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ABSTRACT 
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\ 

This  research  investigated  the  feasibility  of  predicting  a  pilot's  workload  in 
a  single-seat  aircraft.  A  model  was  developed  which  combined  discrete  event 
simulation  output  with  an  existing  workload  methodology  to  predict  workload. 
The  methodology  used  was  the  Subjective  Workload  Assessment  Technique  (SWAT), 
which  recognizes  three  dimensions  of  workload:  time  load,  mental  effort  load, 
and  psychological  stress  load.  Discrete  event  simulation  output  provided 
estimators  for  the  time  load  dimension  while  pilots'  subjective  estimates  were 
used  for  the  other  two  SWAT  dimensions. 

Using  a  queueing  analogy  with  the  pilot  modeled  as  the  server  and  the  pilot's 
tasks  as  customers,  several  modeling  options  were  investigated  in  the  discrete 
event  simulation.  The  main  issue  examined  was  whether  processing  tasks  in 
series  or  parallel  produced  more  accurate  work  load  predictions.  Two  distinct 
simulation  models  were  developed,  each  employing  a  priority  system  with 
preemption.  In  the  parallel  model,  the  number  of  tasks  that  could  be 
processed  simultaneously  was  dependent  on  the  particular  combination  of  tasks 
currently  requiring  pilot  resources.  With  each  task  demanding  specified 
resources,  the  availability  of  pilot  resources  was  determined  by  reference  to 
the  Multiple  Resource  Model.  Three  different  measures  from  the  simulation 
model  were  evaluated  as  surrogates  for  the  SWAT  time  load  dimension:  pilot 
idle  time,  task  interruption  rate,  and  simultaneous  task  rate.  Four  different 
definitions  of  task  interruption  rate  were  also  considered. 

The  predictions  of  the  work  load  model  were  validated  by  comparison  to  SWAT 
work  load  measurements  taken  under  Identical  conditions  in  a  high-fidelity 
flight  simulator.  This  analysis  was  accomplished  using  a  multivariate 
comparison  of  means  and  a  profile  analysis.  Both  techniques  produced  similar 
results.  From  a  model  viewpoint,  little  difference  in  prediction  accuracy 
between  the  serial  and  parallel  simulation  models  was  found  when  pilot  Idle 
time  was  used  as  the  surrogate  predictor.  When  evaluating  predictors,  the 
most  accurate  results  wore  produced  using  pilot  idle  time  and  the  simultaneous 
task  rate  predictors. 
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One  critical  concern  in  each  case  is  the  workload 
imposed  on  the  human  by  the  design  of  the  system.  In  this 
research,  workload  refers  to  the  mental  or  cognitive  load 
on  a  human  rather  than  the  physical  load  that  might  be 
imposed  in  some  situations.  Workload  is  a  concern  as  a  new 
machine  is  designed  and  steps  are  taken  to  alleviate, 
minimize,  or  balance  the  workload  imposed  on  the  human. 
Workload  may  be  identified  through  several  methods,  but 
ultimately,  at  the  present  time,  this  must  involve  the  use 
of  a  mock-up,  prototype,  or  simulation  of  the  real  system 
to  measure  the  workload  imposed  on  the  human  operator. 

A  problem  may  arise  because  any  hardware  which 
reasonably  replicates  the  final  system  is  usually  not 
available  until  late  in  the  development  process  of  that 
system.  Problems  discovered  while  testing  hardware  may 
cause  expensive  redesign  and  considerable  production  delays 
for  that  system.  Clearly,  what  is  needed  is  some  means  of 
assessing  workload  early  in  the  design  phase  of  system 
development. 

Specific  Aims 

-  v 

In  order  to  pursue  this  goal,  this  research  is  limited 

I 

to  the  problem  of  assessing  a  pilot's  workload  while  fjlying 
an  aircraft.  The  intent  is  to  develop  and  validate  a  model 

f 

which  predicts  pilot  workload.  This  effort  will  show  the 
feasibility  of  such  a  model  and  show  that  the  model's 
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y predictions  relate  reasonably  well  to  measured  workload 
levels.  This  study  focuses  on  pilot  workload  in  a  single¬ 
seat,  fighter  aircraft.  While  the  methods  employed  are 
generally  applicable  to  other  human-machine  systems,  it  is 
not  known  if  any  derived  conclusions  can  also  be  applied  to 
other  systems.  This  research  is  unique  in  that  it  employs 
a  model  significantly  different  from  other  existing  models. 
This  model  combines  predictions  from  discrete-event 
simulation  with  subjective  predictions,  while  other  models 
generally  rely  on  a  single  method.  In  addition,  this  model 
employs  the  structure  of  an  existing  measurement 
methodology  to  provide  a  framework  for  these  predictions. 
This  model  may  be  useful  in  early  aircraft  design  stages  to 
assess  the  impact  of  a  particular  aircraft  configuration  on 
pilot  workload.  $*}..*•  A  ^ ^  •  '**'*■"/ 

/  /  / 

Significance  v,i 

The  prediction  of  pilot  workload  is  an  emerging  area 
that  goes  beyond  simply  defining  or  measuring  pilot 
workload.  In  itself*  workload  is  something  of  a  fuzzy* 
intuitive  concept.  Several  popular  definitions  exist*  but 
these  definitions  do  not  necessarily  agree  with  one 
another.  In  spite  of  not  having  a  common  definition* 
workload  measurement  efforts  have  been  on-going  for  a 
number  of  years.  Several  measurement  techniques  have  shown 
reasonable  success  in  discriminating  between  different 


T 


t 


A  METHODOLOGY  FOR 


PREDICTING  PILOT  WORKLOAD 

DISSERTATION 

Presented  in  Partial  Fulfillment  of  the  Requirement  for 
the  Degree  Doctor  of  Philosophy  in  the  Graduate 
School  of  the  Ohio  State  University 

By 

Thomas  Frederick  Schuppe,  B.S.,  M.S. 

*  *  *  *  * 

The  Ohio  State  University 
1988 


Dissertation  Committee: 
R.  A.  Miller 
P.  J.  Smith 
B.  L.  Nelson 


Approved  by 


Adviser 

Department  of  Industrial 
and  systems  Engineering 


88  8  16 


Copyright  by 

Thomas  Frederick  Schuppe 
1988 


To  My  Wife,  Mary 


ACKNOWLEDGMENTS 


I  would  like  to  thank  my  dissertation  committee.  Dr. 

A1  Miller,  Dr.  Phil  Smith,  and  Dr.  Barry  Nelson.  Each 
contributed  immeasurably  to  this  document  and  supported  and 
encouraged  me  throughout  its  preparation.  I'd  also  like  to 
thank  Dr.  Walt  Gif fin  who  guided  me  during  the  early  phases 
of  my  program.  Mr.  Gary  Reid  of  the  Air  Force  Armstrong 
Aerospace  Medical  Research  Laboratory  at  Wright-Patterson 
Air  Force  Base,  Ohio  should  also  be  commended  for  patiently 
explaining  the  SWAT  methodology  and  placing  the  resources 
of  that  laboratory  at  my  disposal.  I'd  also  like  to  thank 
the  pilots  of  the  166  Tactical  Fighter  Squadron,  the  178 
Tactical  Fighter  Group,  and  the  906  Tactical  Fighter  Group 
for  providing  some  of  the  basic  data  that  went  into  this 
research.  Finally,  I'd  like  to  thank  the  multitude  of 
other  researchers,  professors,  and  fellow  students  who 
shared  their  thoughts  and  work  with  me  and  didn't  receive 
enough  credit.  I  couldn't  have  done  this  without  help  form 
all  of  these  people. 


iii 


VITA 


August  24,  1944  . Born,  Fond  du  Lac,  Wis. 

1967  .......  .  B.S. ,  Univ.  of  Wisconsin 

1967  -  1968  . Project  Engineer, 

McDonnell  Aircraft, 

St.  Louis,  Missouri 

1968  -  Present  ...  .  Officer,  United  States 

Air  Force 

1976 . .  M.S.,  Air  Force  Institute 

of  Technology,  Wright- 
Patterson  Air  Force  Base, 
Ohio 


FIELDS  OF  STUDY 

Major  Field:  Operations  Research 

studies  in  stochastic  processes,  mathematical 
programming,  and  statistics 


iv 


TABLE  OF  CONTENTS 


DEDICATION . iii 

ACKNOWLEDGMENTS  .  iv 

VITA . . . V 

LIST  OF  TABLES . ix 

LIST  OF  FIGURES . xi 

CHAPTER  PAGE 

I .  INTRODUCTION  .  1 

Specific  Aims  .......  .  2 

Significance  .  3 

Overview . 5 

II.  BACKGROUND . 8 

Workload  Definition  .  8 

workload  Measurement  .....  .  .,11 

Workload  Prediction . 18 

Pilot  Models . 23 

Summary  . . 28 

III.  METHODOLOGY . . . .  .  28 

Introduction  ..........  .  28 

Definitions  and  Assumptions  .  .  28 

The  Workload  Model . 31 

The  Radar-Aided  Mission/Aircrew 
Capability  Exploration  (RAM/ACE) 

Experiment  .....  .  38 

RAM/ ACE  Replication  .  ....  40 

Experimental  Design  and  Data  Analysis  ....  44 
Summary . 52 


v 


IV.  SIMULATION  MODEL . .53 

Single  Channel  Model  .  55 

Parallel  Processing  Model  .  57 

Task  Inter-Arrival  and  Service  Tine 

Development . 65 

Parallel  Server  Model  Calibration  .  70 

Model  Verification . 71 

Model  Output . 75 

Summary . 78 

V.  RESULTS . 80 

Introduction  .  80 

Computer  Simulation  Results  .  82 

SWAT  Score  Conversion . 84 

Predictor  Comparison  ...  .  86 

Multivariate  Comparison  of  Means  .  88 

Profile  Analysis  .....  .  93 

Sensitivity  of  Results . .  .  103 

Task  Interruption  Definition  .  104 

Workload  Expert's  Transformation  .  .  .  106 

Service  Time  Degradation  Factor  ....  108 

Summary  .  , . 114 

VI.  CONCLUSIONS  AND  RECOMMENDATIONS . 116 

Introduction  . . 116 

Conclusions  .  .......  116 

Workload  Predictors  .  118 

Serial  vs  Parallel  Models  .  . . 120 

Expert  versus  non-expert 

pilot  subjects  ...........  123 

sensitivity  Issues  .  12$ 

General  Conclusion  ..........  127 

Recommendations  . . 128 

Contributions . 131 


APPENDICES 

A.  SWAT  Cards  and  Sorting  Instructions  .....  132 


B.  SWAT  Card  Sort  Results . 151 

C.  Workload  Survey . 159 

D.  SWAT  Workload  Ratings  Summary . 240 

E.  Simulation  Model  Inter-Arrival  Time 

and  Service  Time  Distributions . 244 


vi 


P.  Simulation  Programs  . 

G.  Workload  Transformation  Survey 

and  Results  . 

H.  Workload  Ratings  and  Predictions 

LIST  OF  REFERENCES  . 


LIST  OF  TABLES 


TABLE  PAGE 

1.  Multiple  resource  matrix  .  60 

2.  Degradation  factors  .  61 

3.  Utilization  rate  squared  error, 

parallel  server  model  .  73 

4.  Task  interruption  predictor  definition  .  78 

5.  Workload  ratings  and  predictions  summary  ....  81 

6.  Predictor  values,  single  server  model  .  83 

7.  Predictor  values,  parallel  server  model  ....  83 

8.  Simulation  output,  SWAT  score 

transformations . 85 

9.  Multivariate  comparison  process  .  87 

10.  Multivariate  comparison  of  means  .  91 

11.  Profile  analysis  results,  1982  data . 99 

12.  Profile  analysis  results,  1986  data . 101 

13.  Transformation  sensitivity,  multivariate 
comparison  of  means  (Predictor  »  SS-PIR)  ....  107 

14.  Transformation  sensitivity,  profile 

analysis  (Predictor  ■  SS-PIR)  .  108 

15.  Service  time  degradation  limit  sensitivity  .  .  .  110 

16.  Degradation  limit  sensitivity, 

multivariate  comparison  of  means  .  Ill 

17.  Degradation  limit  sensitivity, 

profile  analysis,  1982  data  .  .....  112 

viii 


■.VUV.V'^  \  v  Vi.y  IV  lyuti*  a  IV  a  vv.ru.> 


18.  Degradation  limit  sensitivity, 

profile  analysis,  1986  data  .....  .  113 

19.  Multivariate  comparison  of  means, 

idle  time  predictor  only  .  . . 121 

20.  Profile  analysis,  idle  time  predictor  only  .  .  .  122 

21.  Card  sort  data . 154 

22.  SWAT  workload  ratings,  time  load 

emphasis  group  . . 155 

23.  SWAT  workload  ratings,  mental  effort 

load  emphasis  group . 156 

24.  SWAT  workload  ratings,  psychological 

stress  load  emphasis  group  .  15? 

25.  SWAT  workload  ratings,  composite 

group  solution . 158 

26.  swat  workload  ratings  (1986) 

ABL  configuration . 241 

27.  SWAT  workload  ratings  (1986) 

TF/TA  configuration . 242 

28.  SWAT  workload  ratings  (1986) 

TTA  configuration . 243 

29.  Pilot  task  summary . 246 

30.  Task/segment  summary,  all  configurations 

mean  inter-arrival  and  service  times  .  250 


31.  Simulation  output,  SWAT  score 

transformation  (all  responses)  .  . . 288 

32.  Simulation  output,  SWAT  score 

transformation  (pilots  only)  .  288 

33.  Simulation  output,  SWAT  score 

transformation  (experts  only)  .  ...  289 

34.  Workload  measurements  and  estimates  .  293 


IX 


LIST  OF  FIGURES 


FIGURES  PAGE 

1.  Workload  dimensional  comparison  ........  30 

2.  Derivation  of  time  load  predictors . 47 

3.  Queueing  network  structure, 

serial  and  parallel  models  .  56 

4.  Task  arrival  process, 

single  server  model  .  58 

5.  Service  completion  process, 

single  server  model  .  59 

6.  Task  arrival  process, 

parallel  processing  model  .  63 

7.  Service  completion  process, 

parallel  processing  model  .  64 

8.  Parallel  server  model  calibration  .  72 

9.  Workload  profiles,  SWAT  and  PRO-SWAT 

(1986,  predictor  »  PRO-SWAT)  .  97 

10.  Workload  profiles,  SWAT  and  PRO-SWAT 

(1982,  predictor  ■  PRO-SWAT)  . . 98 

11.  Generic  air-to-ground  weapon 

delivery  profile  .  166 

12.  Synthetic  aperture  radar  .  175 

13.  DBS  ground  plan . 177 

14.  RAM/ACE  advanced  baseline 

forward  crew  station . 178 

15.  RAM/ ACE  advanced  baseline 

aft  crew  station . 180 


16.  RAM/ACE  forward  crew  stick  functions  .  183 

17.  RAM/ACE  forward  crew  station 

throttle  functions  .  184 

18.  RAM/ACE  aft  crew  station  left 

hand  controller  functions  .  186 

19.  RAM/ACE  aft  crew  station  right 

hand  controller  functions  .  137 

20.  RAM/ACE  up-front  control  .  188 

21.  RAM/ ACE  UFC  IFF  sublevel . 189 

22.  RAM/ACE  UFC  waypoint  sublevel  .  190 

23.  RAM/ACE  waypoint  data  entry  format  .  191 

24.  RAM/ACE  UFC-TF/TA  sublevel  .  192 

25.  RAM/ACE  wide  field-of-viaw  HUD . 194 

26.  RAM/ACE  HUD  symbology . 195 

*7.  RAM/ACE  basic  HUD  display  designated  .  199 

28.  RAM/ACE  display  logic  .  200 

29.  RAM/ACE  menu  format . 201 

30.  RAM/ACE  head-down  display  (HDD)  ........  202 

31.  RAF/ACE  ;.cs  format,  Maverick  weapon  load  .  .  .  204 

32.  RAM/ACE  ACS  format,  WASP  weapon  load . 206 

33.  RAM/ACE  threat  display  .  207 

34.  RAM/ACE  FLIR  display . 208 

35.  RAM/ ACE  radar  real  beam 

ground  map  display . 210 

36.  RAM/ ACE  RBGM  to  EXP1  progression . 212 

37.  RAM/ACE  radar  designation  orocedure  .  214 

38.  RAM/ACE  EHSI  display . 215 

xi 


39.  RAM/ ACE  EHSI  data  sublevel, 

aircraft  option  selected  .  218 

40.  RAM/ACE  EHSI  data  sublevel, 

waypoint  option  selected  .  219 

41.  RAM/ACE  pop-up  command 

symbology  on  the  HUD . 220 

42.  Representative  SAR  imagery  .  222 

43.  Imagery  comparison  .  223 

44.  Typical  FLIR  imagery . 225 

45.  Imaging  infrared  Maverick  .  226 

46.  HR  Maverick . 227 

47.  TF/TA  HUD  format, 

programmed  tactical  corridor  .  232 

48.  TTA  threat  display . 238 


xii 


m 


CHAPTER  I 


INTRODUCTION 


Humans  are  increasingly  being  placed  in  more  demanding 
situations  in  a  variety  of  human-machine  environments. 

This  condition  is  especially  true  in  situations  where  the 
human  acts  as  a  controller  or  operator  of  the  human-machine 
system.  In  order  to  function  properly,  the  human  must  be 
able  to  properly  receive  a  wide  variety  of  inputs,  both 
visual  and  aural,  process  these  inputs  in  a  timely  manner, 
and  respond  in  such  a  way  that  the  human-machine  system 
efficiently  achieves  its  intended  purpose.  This  process  is 
complicated  by  many  factors,  especially  as  the  interactions 
between  the  two  elements  of  this  system,  human  and  machine, 
increase  in  number  and  complexity. 

Several  common  examples  of  such  systems  exist:  a 
pilot  flying  an  aircraft,  an  air  traffic  controller 
directing  air  traffic,  a  nuclear  power  plant  operator 
controlling  his/her  system.  Although  all  of  these  systems 
function  in  different  domains,  the  interactions  between  the 
human  and  the  machine  in  each  case  are  similar  in  several 
ways. 
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levels  of  workload.  However,  workload  measurement  requires 
that  some  physical  system  (aircraft,  flight  simulator,  or 
laboratory  apparatus)  be  available  to  measure  the  workload 
of  a  human  pilot  or  subject.  This  approach  may  be 
acceptable  for  some  purposes,  but,  it  is  not  an  acceptable 
method  to  predict  pilot  workload  during  the  early  design 
phases  of  a  new  aircraft  when  actual  hardware  is  physically 
non-existent. 

To  address  this  problem,  a  workload  modeling  technique 
would  be  helpful.  Several  methods  used  to  predict  pilot 
workload  do  exist.  Some  are  based  on  modeling  techniques 
such  as  simulation  or  time-line  analysis.  These  are 
adequate  to  account  for  timing,  sequencing,  and  rate  change 
issues,  but  are  generally  insensitive  to  human-oriented, 
psychological  issues  (32).  Attempts  have  been  made,  for 
example,  to  include  the  effects  of  stress  on  performance 
within  a  simulation  model,  but  these  are  often  without  a 
theoretical  basis  and  their  accuracy  must  be  questioned. 

Other  predictive  techniques  rely  on  the  subjective 
opinions  of  "expert"  pilots  to  assess  workload.  Battiste 
and  Hart  (2)  and  Reid,  et  al,  (31)  each  describe  a  separate 
method  that  will  predict  workload.  Theoretically,  a  pilot 
skilled  in  a  particular  type  of  aircraft  should  be  able  to 
extrapolate  an  estimate  of  workload  to  a  new,  non-existent 
system  similar  to  the  system  with  which  he  is  familiar. 
While  pilots  have  been  able  to  assess  the  impact  of  mental 
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and  psychological  factors  on  workload,  Battiste  and  Hart 
(2)  have  found  inaccuracies  in  estimating  the  impact  of 
complex  or  unfamiliar  timing  elements.  This  suggests  a 
need  for  some  other  method  to  assess  the  impact  of  task 
timing  and  rate  changes  on  workload. 

Developing  a  predictive  model  which  will  overcome 
these  deficiencies  will  help  reduce  system  cost  and  pilot 
workload  in  future  aircraft  systems.  If  pilot  workload  can 
be  predicted  with  reasonable  accuracy  early  in  the  design 
process,  unacceptable  levels  of  workload  can  be  identified 
and  design  changes  can  be  made  with  relative  ease.  As 
systems  are  now  developed,  the  measurement  of  pilot 
workload  occurs  only  after  hardware  is  produced  and  then 
design  changes  are  both  difficult  and  expensive.  A 
validated  predictive  methodology  would  both  save  the  cost 
of  hardware  design  changes  and  expedite  the  design  process. 

Overview 

The  next  chapter  provides  the  background  for  this 
study.  It  begins  with  a  discussion  of  the  concept  of 
workload  and  then  reviews  recent  efforts  to  measure  and 
predict  workload.  Special  emphasis  is  given  to  the  Radar 
Aided  Mission/Aircrew  Capability  Exploration  (RAM/ACE)  (13 
and  27)  experiment  since  it  bears  directly  on  this 
research . 

The  next  chapter  describes  the  general  method  used  in 
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this  research,  beginning  with  a  discussion  of  specific 
definitions  and  assumptions.  The  general  workload  model  is 
described  next,  with  the  exception  of  the  simulation  model, 
which  is  deferred  to  the  following  chapter.  However,  the 
interface  between  the  workload  model  and  the  simulation 
model  is  described.  A  replication  of  the  original  RAM/ACE 
experiment  is  described  and  used  to  illustrate  the  use  of 
the  workload  model.  Finally,  the  experimental  design  and 
data  analysis  methods  are  described. 

The  next  chapter  describes  the  simulation  models 
developed  in  this  research.  These  simulation  models  form 
only  a  part  of  the  workload  model  but  are  described 
separately  to  allow  those  readers  specifically  interested 
in  simulation  to  direct  their  attention  to  this  chapter. 
Both  versions  of  the  model  (single  server  and  parallel 
server)  are  described  in  detail.  A  discussion  of  input 
data  modeling  is  then  presented,  followed  by  a  description 
of  how  the  input  data  was  adjusted  for  each  model  version 
and  each  aircraft  cockpit  configuration.  This  is  followed 
by  a  description  of  the  process  used  to  verify  the 
simulation  model  and  a  brief  description  of  important  model 
outputs . 

The  next  chapter  is  devoted  to  the  specific  results 
derived  from  this  study.  The  output  from  simulation-based 
production  runs  is  described  and  the  procedure  used  to 
convert  these  output  values  to  the  general  workload  model 


terminology  is  described.  The  workload  estimates  derived 
from  several  different  predictors  are  then  presented. 
Finally,  the  results  of  a  sensitivity  analysis  performed  to 
assess  the  criticality  of  several  assumptions  are 
presented. 

The  last  chapter  presents  conclusions  and 
recommendations  based  on  the  results  of  this  study  effort. 
Also  included  are  several  appendices  which  provide 
supporting  evidence  and  additional  detail.  Of  special 
interest  is  Appendix  C  which  includes  the  instructions  and 
cockpit  equipment  description  for  the  RAM/ACE  experiment 
and  its  replication.  It  is  included  here  to  illustrate  the 
degree  of  detail  available  in  the  experiment  and  provide  a 
reference  since  it  is  not  otherwise  readily  available. 


BACKGROUND 


An  important  design  issue  for  an  aircraft  cockpit  is 
the  amount  of  workload  that  design  will  impose  upon  a  pilot 
or  crewmember.  Recent  advances  in  computers  and  automation 
have  attempted  to  alleviate  a  pilot's  workload,  but  in  many 
cases,  have  simply  refocussed  it.  Formerly,  a  pilot 
functioned  primarily  as  an  active  controller  in  the 
aircraft  system,  but  more  recently  has  assumed  a  less 
active  role,  becoming  more  of  a  monitor  and  decision  maker. 
This  led  to  a  shift  in  a  pilot's  workload  from  physical  to 
mental  workload.  Recognition  of  this  change  is  important 
and  is  the  first  step  in  attempting  to  define,  measure, 
predict,  or  alleviate  a  pilot's  workload. 

Workload  Definition 

Unfortunately,  there  is  little  agreement  on  a 
definition  of  workload,  either  in  a  general  sense,  or 
specifically  related  to  mental  workload.  Several  authors 
have  proposed  definitions,  but  a  great  deal  of  variety  is 
evident  in  these  proposals.  Senders  <36)  defines  workload 
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as  a  non-accessible,  hypothetical,  unidimensional,  internal 
variable  over  the  range  of  0.0  -  1.0  in  any  human  operator. 
In  his  view,  workload  may  be  considered  the  ratio  of  demand 
to  capacity  with  respect  to  any  action.  He  further 
suggests  that  workload  is  a  meaningful  concept  only  in  the 
context  of  a  well-defined  task  which  must  be  performed  to  a 
stipulated  criteria. 

Rouse  (33)  proposes  that  mental  workload  has  two 
dimensions:  fraction  of  attention  and  intensity  of  effort. 
He  further  suggests  that  the  product  of  these  two  might  be 
a  reasonable  composite  definition  of  workload.  While  this 
definition  is  intuitively  appealing,  measuring  workload  in 
both  of  these  dimensions  may  be  a  problem. 

Sheridan  a.id  Simpson  (38)  view  mental  workload  as 
primarily  concerned  with  information  processing  and 
decision  making.  They  suggest  that  mental  workload,  which 
can  only  be  inferred  and  not  directly  measured,  is  some 
combination  of  mental  effort,  information  processing,  and 
emotion  in  response  to  task  demand.  This  leads  to  three 
category  descriptors:  task  time  constraints,  task 
uncertainty  and  complexity  of  planning,  and  task-induced 
psychological  stress. 

Wickens  (46)  defines  workload  in  terms  of  the  relation 
between  resource  supply  and  task  demand.  As  long  as 
resources  are  available,  workload  is  inversely  related  to 
reserve  capacity.  Once  all  resources  are  occupied. 


10 


workload  is  inversely  related  to  the  level  of  task 
performance.  Wickens  agrees  with  Senders  that  workload 
involves  a  relation  between  task  demand  and  the  operator's 
capacity.  However,  Senders  uses  a  single  dimension  model 
while  wickens  emphasizes  a  multiple  resource  model.  This 
leads  to  a  multi-dimensional  concept  for  workload.  Wickens 
also  states  that  workload  can  be  affected  by  changes  in 
operator  capacity  and  changes  in  task  resource  demands. 

Johannsen  (14)  suggests  that  workload  is  an  umbrella 
concept  which  includes  input  load  and  operator  effort. 

Input  load,  which  is  influenced  by  the  environment,  the 
system  design,  and  operating  procedures,  impacts  operator 
effort  and  ultimately  affects  operator  performance. 
Johannsen,  et  al,  (15)  claim  workload  is  multi-dimensional 
but  do  not  specify  what  those  dimensions  are.  They  do 
suggest  that  it  would  be  intuitively  appealing  if  these 
dimensions  could  be  combined  into  a  single  number.  This 
would  allow  comparisons  between  different  measurements,  but 
at  a  loss  of  some  of  the  dimensionality  information. 

It  is  difficult  to  establish  a  concensus  definition 
from  all  of  these  definitions.  It  does  appear  that  work¬ 
load  is  multi -dimensional  in  nature  and  involves  the  rela¬ 
tion  of  task  demands  to  operator  capacity.  The  workload 
dimensions  proposed  by  Sheridan  and  Simpson  (mental  effort, 
information  processing,  and  emotion)  seem  to  be  representa¬ 
tive  of  the  various  dimensions  discussed  by  other  authors. 
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Operator  capacity  may  also  be  multi-dimensional  and  is  a 
variable  influenced  by  many  factors.  Further  details  of 
this  concensus  definition  of  workload  will  be  provided  in 
the  next  chapter. 

Workload  Measurement 

Once  a  definition  of  workload  is  established,  the  next 
step  is  finding  some  means  to  measure  workload,  with  the 
number  of  different  definitions  available  for  workload,  it 
is  not  surprising  that  there  are  even  more  measurement 
techniques.  In  fact,  Moray  (21)  states  that  there  is  no 
agreed-upon  definition  of  mental  workload  and  no  agreement 
on  how  to  measure  it.  The  very  complexity  of  workload  has 
resulted  in  its  measurement  by  many  methods,  but  the 
relationships  among  these  methods  is  not  clearly 
established. 

Several  authors  have  provided  summaries  and  overviews 
of  the  most  well-known  measurement  methods.  Williges  and 
Wierwille  (51)  provide  a  comprehensive  review  of  fourteen 
specific  classes  of  behavioral  workload  measures  grouped 
into  three  general  categories:  Subjective  opinion,  spare 
mental  capacity,  and  primary  task  metrics.  They  conclude 
that,  due  to  the  multi-dimensionality  of  workload,  no 
single  metric  can  be  recommended  as  the  definitive  measure 
of  mental  workload.  Rather,  they  suggest  that  the  most 
promising  assessment  procedure  should  include  multiple 


measures  of  subjective  opinions,  spare  mental  capacity,  and 
primary  "T,3k  measures. 

Wierwille,  Rahimi,  and  Casali  (50)  evaluated  16 
measures  of  mental  workload  using  a  simulated  flight  task 
emphasizing  mediational  (cognitive)  activity.  Their 
results  indicate  that  two  primary  task  measures,  two 
subjective  measures,  one  spare  mental  capacity  measure,  and 
two  physiological  measures  were  reliably  sensitive  to 
mediational  loading.  In  a  similar  study,  Wierwille  and 
' onnor  (49)  evaluated  20  workload  measures  using  a 
psychomotor  task  in  a  moving-base  aircraft  simulator.  They 
showed  that  two  subjective  measures  and  one  primary  task 
measure  demonstrated  sensitivity  to  all  levels  of  load. 

None  of  the  measures  considered  showed  any  intrusion  on  the 
primary  task. 

Casali  and  Wierwille  (4)  performed  a  sensitivity/ 
intrusion  comparison  of  eight  mental  workload  estimation 
techniques  using  e  flight  task  emphasizing  perceptual 
piloting  activities.  They  found  no  evidence  of  intrusion 
and  six  of  the  eight  measures  proved  sensitive  to  the  load. 
The  sensitive  measures  included  two  subjective  rating 
measures,  two  secondary  task  measures,  one  primary  task 
measure,  and  one  physiological  measure.  They  also 
concluded  that  primary  task  measures  are  quite  task- 
specific  and  should  be  used  cautiously. 


Rahimi  and  Wierwille  (28)  looked  at  the  sensitivity 
and  intrusion  of  eight  workload  estimation  techniques  in 


piloting  tasks  emphasizing  mediational  activity.  The 
mediational  activity  employed  in  this  aircraft  simulator 
experiment  was  a  variety  of  navigation  problems  sorted  into 
three  load  levels  based  on  the  number  and  complexity  of  the 
calculations  required  to  solve  them.  They  found  one 
primary  task  measure,  one  subjective  opinion  measure,  and 
one  secondary  task  measure  to  be  sensitive  to  the  load. 
However,  the  secondary  task  measure  was  considered  to  be 
intrusive  on  other  pilot  activities. 

Bortolussi,  Kantowitz,  arid  Hart  (3)  compared  four 
widely  used  methods  developed  to  predict  and  measure  pilot 
workload.  These  measures  were  reaction  time,  time 
production  (secondary  task),  inflight  verbal  estimates,  and 
retrospective  rating  scales  (subjective  opinion) .  Using  a 
flight  simulator,  the  authors  found  all  four  techniques 
were  able  to  distinguish  among  levels  of  scenario 
complexity.  Both  secondary  task  methods  and  the  in-flight 
workload  ratings  were  able  to  distinguish  among  levels  of 
difficulty  for  different  segments  within  the  scenarios. 

All  of  these  studies  demonstrate  the  diversity  of 
available  workload  measurement  techniques  and  provide 
evidence  of  their  relative  merits.  To  summarize,  it 
appears  that  primary  task  measures  produce  reliable  results 
if  the  task  selected  is  closely  related  to  the  dimension  of 


interest.  However,  in  a  complex  flight  scenario,  it  may  be 
difficult  to  identify  a  single  task  which  reflects  the  wide 
variety  of  potential  sources  of  operator  load.  Secondary 
tasks,  designed  to  reflect  spare  mental  capacity,  are  often 
sensitive  to  load  but  can  intrude  on  the  prime  task  of 
flying  the  aircraft.  Secondary  task  measures  may  also  lose 
their  sensitivity  as  th.s  operator  becomes  fully  loaded  due 
to  prime  task  activity.  Subjective  measures  appear  to  be 
sensitive  to  a  wide  range  of  pilot  tasks.  However,  there 
is  some  concern  that  a  subjective  measure,  taken  at  the 
conclusion  of  a  flight,  may  not  accurately  represent  the 
workload  over  the  entire  flight.  Other  means  have  been 
proposed  to  collect  the  subjective  information  during  a 
flight,  but  this  collection  process  itself  may  intrude  on 
the  primary  flight  task.  Physiological  workload  measures 
appear  to  be  very  task-specific.  That  is,  for  certain 
narrowly-defined  tasks,  physiological  measures  appear  to  be 
sensitive.  But  for  a  complex  situation  consisting  of  a 
wide  variety  of  diverse  tasks,  no  single  physiological 
measure  appears  to  be  appropriate. 

Of  all  the  workload  measures  discussed,  the  most 
widely  accepted  type  appears  to  be  subjective  measures. 
Several  authors  have  also  come  to  this  conclusion. 
Johannsen,  et  al,  (15)  state  that  despite  all  the  well- 
known  difficulties  in  the  use  of  rating  scales,  these  must 
be  regarded  as  central  to  any  investigation.  Rouse  (33) 


states  that  some  form  of  subjective  measurement  is 
inevitable  if  mental  workload  is  to  be  assessed.  Williges 
and  Wierwille  (51)  concluded  that  no  single  technique  can 
be  recommended  as  the  definitive  behavioral  measure  of 
operator  workload,  but  probably  the  strongest  research 
support  exists  for  using  subjective  opinions  and  task 
analytic  methods  involving  task  coxnponent/time  summation. 
Finally,  Wierwille  and  Connor  (49)  suggest  that  well- 
designed  subjective  rating  scales  are  among  the  best 
techniques  for  evaluating  psychomotor  load. 

Three  subjective  measurement  techniques  have  shown 
success  in  previous  studies:  a  modified  Cooper-Harper  (5) 
scale,  the  Subjective  Workload  Assessment  Technique  (SWAT), 
and  the  NASA  Bi-Polar  technique.  The  first  of  these  is 
derived  from  the  successful  Cooper-Harper  scale  used  to 
evaluate  aircraft  handling  qualities.  It  uses  a  decision 
tree  format  and  asks  the  pilot  a  sequence  of  questions  to 
lead  him  to  a  logical  evaluation  of  handling  qualities. 
Sheridan  and  Simpson  (38)  have  modified  this  scale  to 
address  pilot  workload  and  have  suggested  that  three 
separate  scales  be  created  with  each  scale  addressing  a 
different  dimension  of  workload.  The  dimensions  they 
suggest  are  time  load,  uncertainty  and  planning  load,  and 
mental  stress  load. 

Stein  and  Rosenberg  (40)  have  developed  a  one¬ 
dimensional  version  of  this  scale  designed  to  be  used  at 


one-minute  intervals  during  a  specific  task.  At  the  sound 
of  a  tone,  an  operator  is  asked  to  press  one  of  ten  buttons 
relating  to  how  hard  the  operator  feels  he  is  working  at 
that  time.  A  "1"  would  indicate  essentially  no  effort  is 
required  and  a  "10"  would  indicate  that  the  workload  is 
overwhelming  even  when  maximum  effort  is  used.  This  method 
collects  a  large  amount  of  data  by  sampling  at  one-minute 
intervals  and  strives  to  eliminate  error  caused  by  the 
delay  between  workloading  and  the  assessment  of  that 
workload  after  a  flight  is  complete.  However,  requiring  an 
operator  to  respond  with  a  workload  estimate  once  each 
minute  is  somewhat  intrusive  on  his  primary  task  and  may 
itself  contribute  to  total  workload.  This  technique  is 
fairly  new  and,  while  initial  test  results  appear  to 
accurately  measure  workload,  further  testing  is  required 
for  full  acceptance. 

The  Subjective  Workload  Assessment  Technique  (SWAT) 
developed  by  Reid,  et  al,  (29)  is  also  based  on  concepts 
proposed  by  Sheridan  and  Simpson  (38) .  However,  their 
three  dimensions  of  workload  have  been  modified  to  time 
load,  mental  effort  load,  and  psychological  stress  load. 
Each  dimension  is  considered  to  have  three  levels:  high, 
medium,  or  low.  This  leads  to  a  total  of  27  (3  x  3  x  3) 
possible  workload  ratings  for  a  given  test.  The  method 
itself  requires  two  stages.  In  the  first  stage,  prior  to 
any  actual  workload  assessment,  a  subject  is  asked  to  order 
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the  27  possible  workload  states  from  easiest  to  most 
difficult.  This  order  is  subsequently  used  to  provide  an 
ordinal  scale  for  combined  workload  ratings  through  a 
technique  called  conjoint  measurement  (25) .  In  the  second 
stage  a  subject  performs  some  task  or  operation  and,  at  its 
completion,  provides  a  rating  of  low,  medium,  or  high  in 
each  of  the  three  SWAT  dimensions.  These  three  ratings  are 
then  combined  through  the  conjoint  measurement  technique  to 
yield  a  single  workload  measure.  This  technique  has  been 
tested  extensively  and  has  been  accepted  as  a  reasonable 
measurement  device  for  mental  workload  (13,  27,  42). 

The  NASA  Bi-Polar  measurement  technique  [Hart,  et  al, 
(12)  and  Kantowitz,  et  al,  (18)]  has  also  demonstrated 
reasonable  accuracy  as  a  subjective  workload  assessment 
technique.  This  technique  requires  an  operator  to  complete 
bi-polar  rating  scales  for  ten  items  after  a  set  of  tasks 
is  complete.  Each  operator  is  also  required  to  complete  a 
pairwise  ranking  of  all  ten  items  to  establish  weights  for 
each  item.  By  combining  bi-polar  workload  ratings  with 
their  appropriate  weights,  a  single  metric  of  workload  is 
determined. 

Vidulich  and  Tsang  (42)  found  in  a  laboratory 
experiment  that  the  NASA  Bi-Polar  method  was  successful  in 
measuring  the  differences  in  task  difficulty  as  indicated 
by  a  multivariate  analysis  of  performance*  They  also  found 
a  remarkably  similar  workload  assessment  when  the  SWAT 


technique  was  used  in  the  same  laboratory  test.  Vidulich 
and  Tsang  concluded  that  the  subjective  experience  of 
workload  is  sufficiently  robust  to  be  resistant  to 
variations  in  the  measuring  technique. 

Workload  Prediction 

As  a  general  concept,  workload  prediction  involves 
using  some  method  or  technique  to  forecast  what  an 
operator's  workload  will  be  in  a  "real"  environment.  These 
predictive  techniques  may  vary  from  a  sophisticated  flight 
simulator,  used  to  predict  pilot  workload  for  an  almost 
identical  aircraft,  to  a  simple  "back-of-the-envelopeM 
method  used  for  the  same  purpose.  In  this  study,  a 
distinction  will  be  drawn  between  workload  prediction  and 
workload  measurement.  Measurement  will  denote  any 
laboratory  or  flight  simulator  experiments  in  which  the 
performance  of  a  human  subject  is  actually  measured. 
Prediction  will  denote  all  other  methods  of  estimating 
actual  workload,  including  discrete  event  simulation,  time- 
line  analysis,  and  human  estimates.  This  study  will 
primarily  attempt  to  develop  a  predictive  technique  and 
will  use  an  accepted  measurement  technique  as  a  tool  to 
validate  predictions. 

Rouse  (33)  makes  this  same  distinction  in  discussing 
models  and  measures.  In  his  view,  a  model  of  mental 
workload  is  some  procedure  for  predicting  workload  based  on 


a  priori  conditions.  On  the  other  hand,  a  measure  of 
mental  workload  is  some  measurable  quantity  that  allows  one 
to  determine  the  presence  of  some  specific  level  of 
workload.  From  a  system  design  perspective,  a  model  is 
used  to  predict  how  (usually  physically  non-existent) 
alternate  designs  will  perform  while  measures  are  used  to 
evaluate  designs  once  they  have  resulted  in  a  particular 
system  being  produced.  Thus,  a  measure  of  mental  workload 
would  be  useful  to  validate  the  predictive  ability  of  a 
model  of  mental  workload. 

Several  models  used  to  predict  mental  workload  already 
exist.  Senders  (37)  uses  an  information  theory  model  to 
predict  workload  in  monitoring  and  controlling  situations. 
His  model,  which  assumes  that  a  pilot  always  works  to  full 
capacity,  yields  fraction  of  attention  metrics  in  a  direct 
manner.  Where  this  model  has  been  applied  to  relatively 
simple  situations,  it  has  had  reasonable  success.  However, 
due  to  its  rigid  structure,  it  cannot  be  applied  readily  to 
more  realistic,  complex  tasks. 

Control  theory  models  have  been  used  to  model  tasks 
involving  dynamic  systems.  However,  this  is  only  a  small 
subset  of  the  tasks  for  which  workload  predictions  are  of 
interest,  so  this  model  is  rather  limited  in  application. 

Queueing  theory  overcomes  some  of  the  problems  of  the 
other  two  models.  It  ignores  many  details  of  a  task  and 
instead  concentrates  on  the  time  involved  in  performing  the 


task.  This  allows  a  greater  variety  of  tasks  to  be 
addressed.  Queueing  models  usually  assume  that  task 
demands  occur  randomly  and  each  task  is  accomplished 
perfectly.  Therefore,  levels  of  performance  are  not 
usually  considered,  performance  is  usually  assumed  to  be 
perfect.  This  assumption  may  be  the  major  disadvantage  of 
these  models.  Queueing  models  inherently  produce 
predictions  of  server  utilization. 

Battiste  and  Hart  (2)  describe  a  laboratory  experiment 
in  which  a  workload  measurement  technique  was  used  to 
derive  predictions.  Each  experimental  subject  was 
presented  a  basic  scenario  which  involved  the  simulated 
processing  of  five  different  task  types.  After  an  hour  of 
practice  at  the  basic  scenario,  each  subject  rated  his 
workload  using  the  NASA  Bi-Polar  rating  method.  Following 
this  practice  session,  fifteen  additional  scenarios  of 
increasing  levels  of  difficulty  were  presented.  Prior  to 
each  scenario,  a  description  of  that  scenario  and  a 
schedule  of  task  arrival  times  were  provided  to  each 
subject.  Operators  predicted  the  expected  workload  after 
studying  this  information,  again  using  the  bi-polar  method. 
They  also  rated  their  experiences  at  the  end  of  each 
scenario. 

From  this  experiment,  Battiste  and  Hart  found  that  in 
low  workload  scenarios,  operators  tended  to  overestimate 
the  workload  of  unfamiliar  task  elements  by  a  large  margin. 


In  more  complex  scenarios  (which  also  imposed  more 
workload) ,  operators  predicted  workload  with  greater 
accuracy.  The  authors  concluded  that  operators  could 
predict  the  workload  of  realistically  complex  tasks  if: 

(a)  they  are  familiar  with  the  basic  system,  and  (b)  the 
design,  functional  requirements,  and  operational  procedures 
of  the  proposed  modifications  are  described  clearly. 
Operators  are  less  able  to  predict  unfamiliar  rate  or 
schedule  complexity  manipulations,  for  which  timing  is  an 
important  element.  While  this  method  was  not  entirely 
successful  in  predicting  workload,  their  recognition  of  the 
problem  of  operators  having  difficulty  predicting  the 
timing  of  events  was  significant. 

Reid,  et  al,  (31)  developed  a  predictive  workload 
technique  based  on  his  SWAT  measurement  technique.  In  this 
method,  each  of  a  group  of  experts  is  given  a  detailed 
description  of  a  scenario,  aircraft  equipment,  and  flight 
requirements.  Based  on  this  information,  each  expert 
provides  workload  estimates  in  each  of  the  three  SWAT 
dimensions.  These  ratings  are  then  combined  using  the 
usual  conjoint  measurement  process  to  reach  a  single 
workload  metric.  In  one  study,  this  metric  was  found  to 
have  a  correlation  of  greater  than  0.75  with  SWAT  ratings 
given  after  flight  simulator  missions  reflecting  identical 
flight  conditions.  Although  relatively  new  and  not 
thoroughly  validated,  this  predictive  technique  shows 


promise.  Additional  work  must  be  done  to  validate  this 
technique  over  a  wide  variety  of  workload  situations. 

Eggleston  and  Quinn  (7)  did  a  more  detailed  study  of 
this  projective  SWAT  technique.  They  used  three  variations 
of  the  technique  and  compared  the  results  against  SWAT 
measurements  of  the  same  systems  and  scenarios.  The 
results  of  all  analyses  showed  correlation  coefficients  of 
0.55  to  0.85  between  SWAT  projections  and  measurements. 

The  authors  considered  this  strong  support  for  the 
technique,  but  suggest  further  effort  is  required  for 
validation.  Furthermore,  they  provided  suggestions  for 
developing  adequate  descriptive  material,  selecting  an 
appropriate  definition  of  an  event,  and  determining  the 
desired  knowledge  base  of  a  rater.  Since  they  bear 
directly  on  this  research,  Eggleston  and  Quinn's  efforts 
are  discussed  in  much  greater  detail  in  Chapter  III. 

Based  on  the  predictive  techniques  described  here,  it 
appears  that  workload  prediction  is  possible.  However, 
these  techniques  are  all  relatively  unproven  and  will 
require  further  work  for  validation.  The  next  section 
proposes  a  predictive  technique  that  combines  certain 
elements  of  these  techniques  with  several  other  concepts. 

A  proposed  method  of  validating  this  technique  will  also  be 
described. 
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Pilot  Models 

A  number  of  models  do  exist  which  attempt  to  represent 
a  pilot  accomplishing  normal  tasks  in  a  cockpit.  These 
models  often  differ  in  purpose  so  it  is  not  surprising  that 
differences  can  be  found  in  their  assumptions,  structure, 
and  implementation.  Wickens  (48)  provides  an  excellent 
over-view  of  these  models  and  summarizes  their  strengths  and 
weaknesses.  Because  his  findings  are  so  germane  to  this 
analysis,  they  will  be  discussed  in  some  detail. 

Wickens  divides  these  models  into  two  basic 
categories:  those  that  assume  a  serial  mode  of  processing 
and  those  that  assume  some  form  of  time  sharing  or  parallel 
processing  of  tasks.  Each  of  these  categories  of  models 
can  be  further  subdivided  into  a  model  class  addressing 
allocation  of  resources  and  another  class  referring  to  the 
sources  of  variance  in  competition  between  tasks.  The 
former  refers  to  the  selection  aspects  of  attention  while 
the  latter  addresses  the  scarcity  of  resources. 

Several  serial  processing  models  are  pertinent  to  this 
research.  These  models  assume  that  tasks  are  accomplished 
one  at  a  time.  When  two  or  more  tasks  compete 
simultaneously  for  an  operator's  attention,  some  rule 
determines  which  task  receives  priority  and  which  task  must 
wait.  These  rules  vary  between  models.  For  example,  the 
Human  Operator  Simulator  (HOS)  is  based  on  user-defined 
priorities  (43) . 


The  SAINT  (52)  network  programming  language  has  been 
applied  only  to  discrete  tasks.  This  is  also  true  of  the 
human  operator  model  developed  by  Siegel  and  Wolf  (39) . 
While  most  serial  models  recognize  task  conflict  by  the 
delay  induced  until  a  task  reaches  the  head  of  a  queue, 
Siegel  and  Wolf  take  a  different  approach.  They  determine 
a  workload  estimate,  based  on  the  ratio  of  time  for  task 
completion  to  time  available,  to  determine  a  mean 
completion  time  for  tasks  and  the  task's  probability  of 
errorless  completion. 

wickens  (48)  notes  that  all  of  these  models  share  a 
common  concern  for  the  role  of  time.  Time  becomes  the  only 
resource  of  concern  and  completion  time,  or  task  length, 
defines  the  difficulty  of  a  task.  Early  versions  of  these 
models  generally  do  not  recognize  intensive  aspects  of  task 
demand  or  degrees  of  similarity  between  tasks.  Therefore, 
these  serial  models  choose  to  ignore  evidence  of  parallel 
processing  (45) .  It  is  not  immediately  apparent  if  this  is 
a  serious  drawback  to  this  class  of  models.  However,  it  is 
intuitively  disconcerting.  One  of  the  objectives  of  this 
research  is  to  examine  several  serial  models  and  see  if 
there  appears  to  be  any  general  problem  with  this  approach. 

Several  parallel  processing  models  also  exist. 

Wickens  (48)  states  that  these  models  assume  parallel 
processing  between  tasks  is  ongoing  and  interference 
effects  result  from  competition  for  something  more  than 


tine.  This  additional  aspect  of  competition  is  related  to 
intensity.  A  revised  version  of  HOS  allows  parallel 
processing.  However,  there  is  no  mechanism  for  specifying 
interaction  between  parallel  tasks,  so  all  parallel 
processing  is  assumed  to  be  perfect  processing. 

A  revised  version  of  SAINT  called  MICROSAINT  (20) 
recognizes  eight  demand  levels  in  addition  to  task  time 
requirements.  It  also  recognizes  four  channels  (visual, 
auditory,  cognitive,  and  psychomotor)  for  task  demands  in 
which  interference  may  occur.  The  A3I  model  by  Corker,  et 
al  (6)  makes  similar  assumptions  about  the  four  channels. 
However,  both  of  these  last  two  models  are  limited.  First, 
both  fai)  to  acknowledge  difficulty  variation  of  tasks 
within  a  level.  Second,  the  assumption  of  parallel 
processing  between  channels  appears  to  be  unwarranted. 
Wickens  (45)  shows  there  is  clear  evidence  that  auditory 
and  visual  tasks  interfere,  as  do  perceptual  (both  auditory 
and  visual)  and  cognitive  ones.  In  general,  there  appears 
to  be  little  effort  to  validate  these  models,  so  some 
caution  must  be  exercised  in  their  use. 

Wickens  (46)  suggests  that  models  developed  in 
psychological  laboratories  may  be  useful  to  improve  the 
simulation  models  described  earlier.  He  suggests  his 
multiple  resources  model  (46)  has  received  a  great  deal  of 
validation  and  may  be  appropriate  for  such  a  purpose.  This 
model,  which  is  explained  in  greater  detail  later,  assumes 


that  two  tasks  will  interfere  to  the  degree  that  component 
tasks  are  more  difficult  (demand  more  resources)  and  that 
components  compete  for  overlapping  resources.  North  (23) 
has  developed  a  conflict  matrix  which  estimates  the  degree 
of  task  interference  based  on  the  degree  of  overlap  of  the 
resource  requirements  of  both  tasks.  Use  of  this  matrix 
allows  a  model  to  more  accurately  assess  the  degree  of  task 
interference  between  both  similar  and  dissimilar  tasks. 
Wickens  (48)  recommends  that  the  logic  of  North's  conflict 
matrix  be  incorporated  into  any  model  used. 

To  develop  an  improved  pilot  model,  Wickens  (48) 
recommends  three  possible  approaches.  First,  more 
quantitative  elements  could  be  built  into  a  multiple 
resources  model.  Second,  a  class  of  models  known  as 
multichannel  detection  and  recognition  could  be  extended. 
Finally,  attempts  should  be  made  to  determine  how 


accurately  complex  performance  can  be  predicted  by  serial 
models  with  assumptions  of  single  task  neglect.  As  a  final 
note,  Wickens  suggests  that  it  may  be  necessary  to  accept 


CHAPTER  III 


METHODOLOGY 

Introduction 

This  chapter  describes  the  methodology  used  to  conduct 
this  research.  Specific  definitions  and  assumptions  used 
in  this  study  are  presented  and  discussed.  Two  candidate 
models  are  described  in  detail  and  the  derivation  of 
specific  predictors  is  also  described.  Model  and  predictor 
accuracy  are  determined  by  comparing  model-based 
predictions  of  pilot  workload  with  workload  ratings 
determined  from  a  flight  simulator.  This  comparison 
process  forms  the  basis  of  a  validation  procedure  for  these 
models.  Finally,  the  experimental  design  and  plan  for  data 
analysis  is  presented. 

Definitions  and  Assumptions 

This  research  effort  will  assume  the  workload  concept 
proposed  by  Sheridan  and  Simpson  (38)  and  used  by  Reid,  et 
al,  (29)  in  their  Subjective  Workload  Assessment  Technique, 
is  an  acceptable  definition  of  mental  workload.  Recall 
that  Reid  defined  workload  as  the  demand  on  a  pilot's 
resources  induced  by  a  combination  of  task  time  load, 


mental  effort  load,  and  psychological  stress  load. 

According  to  Potter  and  Acton  (26)  time  load  refers  to  the 
degree  of  task  overlap  or  interruption.  Mental  effort  load 
refers  to  the  amount  of  attention  or  concentration  required 
for  task  performance.  Finally,  psychological  stress  load 
refers  to  the  combined  emotional  and  physical  factors,  such 
as  anxiety  or  confusion,  which  can  otherwise  affect 
subjective  load. 

Other  definitions  and  workload  concepts  were  discussed 
in  Chapter  II.  The  obvious  question  is:  Why  choose  this 
three-dimensional  definition  of  workload  over  the  others? 
Rouse  (33)  described  workload  as  having  two  dimensions  and 
Hart's  Bi-Polar  technique  (12)  employs  nine  dimensions. 
While  these  may  seem  to  be  radically  different  definitions, 
they  appear  to  be  relatable  as  shown  in  Figure  1. 
Essentially,  this  figure  shows  that  the  Reid  definition 
corresponds  to  a  breakdown  of  Rouse's  definition.  Hart's 
definition  appears  to  be  a  further  breakdown  of  Reid's. 

As  Figure  1  suggests,  adding  additional  dimensions  did 
not  add  another  aspect  to  mental  workload,  it  simply  sub¬ 
divided  those  dimensions  already  identified.  This  may  add 
some  clarity,  but  it  does  not  add  new  dimensions  to  the 
workload  concept.  Rouse's  two-dimensional  concept  differs 
from  Reid's  three-dimensional  concept  by  dividing  Intensity 
of  Effort  into  Mental  Effort  Load  and  Psychological  Stress 
Load.  This  enables  a  somewhat  clearer  understanding  of 
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Rouse's  dimension  of  Intensity  of  Effort.  Furthermore,  the 
substantial  amount  of  evidence  supporting  SWAT  provides 
justification  for  using  this  definition  over  Rouse's 
definition. 

Expanding  the  three-dimensional  definition  to  nine 
dimensions  also  adds  some  clarity,  but  its  benefits  must  be 
questioned.  Vidulich  and  Tsang  (42)  showed  that  SWAT  and 
the  NASA  Bi-Polar  technique  produced  equally  valid  results 
and  only  cited  ease-of-use  differences  as  reasons  for 
choosing  one  over  the  other.  Potter  and  Acton  (26)  also 
found  differential  sensitivity  between  the  three  SWAT 
subscales,  supporting  the  assumption  that  the  three 
dimensions  of  SWAT  represent  separate  dimensions  of 
subjective  load. 

For  these  reasons,  this  research  will  use  the  three- 
dimensional  concept  of  workload  proposed  by  Sheridan  and 
Simpson  (38),  and  refined  in  the  development  of  the  SWAT 
measurement  technique  by  Reid,  et  al  (29). 

The  Workload  Model 

The  model  proposed  for  this  research  effort  will  be 
patterned  on  the  three-dimensional  workload  definition  just 
described.  Separate  predictions  will  be  made  in  each 
dimension  and  their  combined  effect  determined  using  the 
conjoint  measurement  technique  employed  in  SWAT.  This 
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combined  metric  will  constitute  the  primary  prediction  of 
the  model. 

A  study  by  Battiste  and  Hart  ( 2 ) ,  previously  described 
in  Chapter  II,  concluded  that  operators  are  less  able  to 
predict  unfamiliar  rate  or  schedule  complexity 
manipulations  for  which  timing  is  an  important  element. 

This  suggests  that  a  purely  subjective  prediction  of  time 
load  may  be  inaccurate  and  some  alternate  means  should  be 
found  to  predict  workload  in  the  time  load  dimension.  To 
accomplish  this,  workload  in  the  time  load  dimension  will 
be  predicted  using  a  discrete  event  simulation  model. 

The  development  of  such  a  model  requires  that  many 
assumptions  be  made.  A  major  assumption  of  this  work  is 
that  a  queueing  model  is  a  reasonable  representation  of  a 
pilot  in  this  environment.  This  model  views  the  pilot  as  a 
server  in  a  queueing  system  and  the  tasks  he  accomplishes 
as  customers  that  arrive  and  queue  for  service.  This  basic 
structure  is  employed  in  this  research  with  several 
variations.  One  of  these  variations  addresses  the  question 
of  whether  the  pilot  should  be  viewed  as  a  single  server 
processing  tacks  one  at  a  time  in  series,  or  as  a  parallel 
processor,  capable  of  accomplishing  more  than  one  task  at  a 
time. 

Much  discussion  appears  in  the  literature  regarding 
the  concept  of  serial  and  parallel  processing  of  tasks  by 
humans.  It  is  not  the  intent  of  this  research  to  determine 
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whether  such  tasks  are  actually  processed  in  series  or 
parallel.  Rather,  its  purpose  is  to  determine  which  model, 
a  single  server  queueing  model  or  a  parallel  server 
queueing  model,  is  more  accurate  for  such  a  situation. 

The  choice  of  simulation  as  a  means  of  predicting 
workload  in  this  situation  has  several  advantages. 
Prediction  of  workload  involves  modeling  a  complex  human- 
machine  system.  Ideally,  a  set  of  analytic  equations  could 
be  developed  and  solved  to  provide  such  a  prediction. 
However,  it  is  extremely  difficult,  if  not  impossible,  to 
find  analytic  solutions  for  systems  with  complex  sets  of 
inputs  and  outputs  such  as  those  associated  with  a 
pilot/aircraft  system.  Rouse  (31)  recognizes  this 
complexity  and  suggests  that  it  is  often  necessary  to 
resort  to  simulation  to  solve  these  equations. 

Additional  support  for  the  use  of  simulation  comes 
from  the  ease  with  which  simulation  employs  queueing 
models.  As  stated  in  the  Introduction,  a  queueing  analogy 
would  be  used  as  the  basic  means  of  addressing  this  man- 
machine  problem.  Rouse  (31)  supports  this  idea  and  states 
that: 

"Queueing  theory  models  of  human-machine  systems  view 
the  human  as  a  time-shared  computer,  allocating  attention 
and  resources  among  a  variety  of  tasks.  ...  In  multitask 
situations,  the  human  must  allocate  attention  among  tasks. 
Adding  differing  priorities  among  tasks  as  well  as 
different  inter-arrival  time  and  service  time  distributions 
results  in  a  multitask  situation  particularly  amenable  to 
queueing  theory  formulation." 


This  statement  exactly  describes  the  situation 
addressed  in  this  research.  It  further  suggests  that  a 
queueing  model  implemented  in  a  simulation  would  satisfy 
many  of  the  modeling  constraints  of  this  complex  man- 
machine  system. 

While  use  of  a  queueing  model  in  this  manner  will 
yield  information  regarding  how  long  a  task  must  wait  for 
service,  how  busy  the  server  is,  or  how  long  each  task  is 
serviced,  such  a  model  will  not  tell  how  well  such  a  task 
is  performed.  In  fact,  there  is  an  inherent  assumption 
that  each  task  is  performed  correctly  or  acceptably  at  all 
times.  A  modeler  may  add  features  to  a  simulation  which 
recognize  errors  and  require  incorrectly  performed  tasks  to 
be  reaccomplished,  but  this  is  not  inherent  in  the  model. 

In  the  simulation  model  developed  in  this  research, 
the  pilot  will  be  represented  as  a  server  and  will 
accomplish  tasks  normally  associated  with  pilot  duties  (11, 
19,  41).  Tasks  will  arrive  for  service  with  inter-arrival 
times  specified  by  probability  distributions  and  will  wait 
in  a  queue  if  the  pilot  is  occupied  with  another  task  of 
equal  or  higher  priority.  For  the  purpose  of  this  model,  a 
task  is  defined  as  an  action  required  of  the  pilot  to 
maintain  flight  and  accomplish  a  mission,  resulting  from 
information  received  by  the  pilot. 

Two  basic  versions  of  this  simulation  model  will  be 
developed.  The  first  will  model  the  pilot  as  a  single 
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channel  server,  processing  tasks  in  series.  The  second 
version  will  view  the  pilot  as  a  multi-channel  server, 
using  Wickens'  (46)  multiple  resource  concept  to  define  the 
available  channels.  This  multiple  resource  model  allows 
the  possibility  of  more  than  one  task  being  accomplished  at 
any  one  time.  The  model  assumes  that  each  task  requires 
specific  pilot  resources  for  completion.  As  long  as  the 
resource  demands  of  two  separate  tasks  do  not  overlap  too 
severely,  they  may  be  accomplished  simultaneously,  but  with 
an  increase  in  service  times  compared  to  accomplishing 
these  tasks  individually.  If  some  of  the  same  resources 
are  required,  then  service  times  for  each  task  are 
increased  even  more.  Chapter  IV  contains  a  detailed 
description  of  this  simulation  model. 

The  method  used  to  determine  the  availability  of 
resources  will  be  Wickens'  (46  and  47)  multiple  resource 
model.  This  model  postulates  that  humans  possess  several 
different  capacities  with  resource  properties.  Task 
interference  will  occur  if  resources  are  shared,  but  little 
interference  will  occur  if  different  tasks  require 
different  resources.  Wickens'  model  describes  three 
resource  dimensions  with  binary  properties:  input  modality 
(visual  or  auditory),  processing  stages  (encoding  and 
central  processing  or  responding) ,  and  processing  codes 
(spatial  or  verbal).  This  results  in  eight  (2x2x2) 
possible  combinations  that  may  be  demanded  by  a  task. 


In  each  version  (single  channel  server  or  parallel 
processing  model)  of  this  simulation  model,  a  priority 
system  controlled  which  tasks  were  serviced  first. 

Sheridan  and  Simpson  (172)  and  Katz  (93)  define  a  pre¬ 
emptive  priority  system  that  categorizes  pilot  tasks  into 
three  groups: 

1.  Operating  tasks  -  those  concerned  directly  with 
the  operation  of  the  aircraft  which  must  be  handled 
immediately.  Examples  include  aircraft  maneuvering  and 
control,  and  monitoring  radio  communications. 

2.  Monitoring  tasks  -  those  that  may  be  delayed  for  a 
short  period  of  time  while  operating  tasks  are  being 
performed.  Examples  include  systems  monitoring,  out-the- 
window  visual  scan,  and  instrument  cross-checking. 

3.  Planning  tasks  -  tasks  deferrable  into  idle 
periods  of  time.  Examples  include  retrieving  weather 
forecasts,  planning  approaches,  and  computing  landing  data. 

This  preemptive  priority  system  was  employed  in  the 
simulation  model  of  this  research.  The  primary  output  of 
this  simulation  model  was  a  workload  prediction  in  the  time 
load  dimension.  Reid,  et  al,  (28)  suggest  that  time  load 
is  characterized  by  the  number  of  task  interruptions,  the 
number  of  times  tasks  are  performed  simultaneously,  or  the 
amount  of  pilot  spare  time.  Each  of  these  represents  a 
slightly  different  aspect  of  time  load.  Since  it  was 
impossible  to  determine  beforehand  which  is  more 


appropriate,  all  three  were  computed  and  tested  in  a 
validation  process  to  determine  which  was  the  better 
predictor.  In  the  case  of  the  single-channel  model, 
simultaneous  performance  of  tasks  was  not  considered,  nor 
was  task  interruption  rate  considered  for  the  parallel 
processing  model.  Chapter  IV  describes  these  predictors  in 
greater  detail. 

The  purpose  of  developing  these  three  simulation-based 
predictors  was  to  use  them  as  surrogates,  or  replacements 
for  a  subjective  prediction  in  the  Time  Load  dimension. 

This  prediction,  along  with  subjective  predictions  in  the 
Mental  Effort  Load  and  Psychological  Stress  Load  dimension, 
were  obtained  in  two  separate  experiments.  The  first  of 
these  was  accomplished  in  1982  as  part  of  the  Radar  Aided 
Mission/Aircrew  Capability  Exploration  (RAM/ACE)  project 
conducted  by  the  Aerospace  Medical  Research  Laboratory  at 
Wright-Patterson  Air  Force  Base  (7,  13,  and  27).  The 
second  set  of  subjective  predictions  were  gathered  in  1986 
as  part  of  this  research.  Both  of  these  experiments  are 
described  later  in  this  chapter. 

By  using  several  combinations  of  subjective 
predictions  and  discrete-event,  simulation-basAj 
predictors,  it  is  possible  to  develop  sevar  different 
workload  model  predictors  for  each  set  of  experimental 
measurements  taken.  Since  two  sets  of  workload 
measurements  were  taken  (1982  and  1986),  this  would  result 


in  two  sets  of  workload  measurements  for  each  scenario. 

The  details  of  each  of  the  seven  predictors  and  the  methods 
used  in  comparing  these  predictors  to  the  workload 
measurements  are  presented  later  in  this  chapter  in  a 
section  entitled  Experimental  Design. 

The  Radar-Aided  Mission/Aircrew  Capability  Exploration 
(RAM/ACE)  Experiment 

This  study  effort  was  sponsored  by  the  Air  Force 
Aerospace  Medical  Research  Laboratory  (7,  13,  and  27).  Its 
purpose  was  to  explore  several  technological  enhancements 
for  application  in  a  post-1995  time  frame  to  improve  the 
effectiveness  of  fighter  aircraft  in  a  tactical,  air-to- 
ground  environment.  Using  an  advanced  version  of  an  F-15C 
flight  simulator,  each  of  several  enhancements  were 
evaluated  over  a  multitude  of  mission  and  scenario 
combinations  toy  highly  experienced  Tactical  Air  Command  F- 
15  pilots.  Each  of  these  enhancements  was  evaluated 
according  to  several  different  measures  of  merit: 
survivability,  target  kills,  cost,  and  pilot  workload.  An 
analysis  based  on  these  measures  was  then  used  to  recommend 
promising  enhancements  for  future  development  of  air-to- 
ground  aircraft  systems. 

To  determine  aircrew  workload,  the  RAM/ACE  study  used 
the  Subjective  Workload  Assessment  Technique  (SWAT) ,  which 
was  previously  described.  The  RAN/ACE  program  utilized 


this  technique  in  two  ways.  Prior  to  a  pilot  entering  a 
flight  simulator  to  fly  a  specific  mission,  each  pilot  made 
a  subjective  estimate  of  his  expected  workload  using  the 
SWAT  methodology.  This  estimate  was  based  entirely  on  a 
written  description  of  the  mission  requirements,  scenario, 
and  particular  enhancement  being  considered,  with  his 
extensive  knowledge  and  accumulated  experience,  each  pilot 
was  asked  to  integrate  all  pertinent  factors  to  develop  an 
accurate  "prediction”  of  his  own  workload  under  a  specific 
set  of  conditions. 

After  completing  this  predictive  exercise,  the  pilot 
flew  the  same  mission  scenario  in  a  flight  simulator,  which 
also  was  modified  to  have  an  operable  version  of  the 
enhancement  being  considered.  At  the  conclusion  of  the 
flight  simulator  exercise,  each  pilot  made  a  second  set  of 
SWAT  workload  ratings  based  on  his  perceived  workload  while 
flying  the  flight  simulator.  Because  SWAT  is  widely 
accepted  as  a  validated  workload  measurement  technique, 
these  simulator-based  workload  ratings  were  accepted  as 
accurate  measures  of  pilot  workload  for  the  particular 
mission,  scenario,  and  equipment  enhancement  combination 
considered. 

Eggleston  and  Quinn  (7)  describe  how  they  compared  the 
SWAT  predictions,  which  they  called  PRO-SWAT,  with  the  SWAT 
measurements.  Although  they  expressed  some  cautions  about 
using  the  PRO-SWAT  method,  they  did  find  that  the  actual 


correlation  between  PRO-SWAT  predictions  and  SWAT 
measurements  ranged  from  0.55  to  0.85.  While  these  results 
seemed  encouraging,  it  was  difficult  to  assess  the 
reliability  and  validity  of  the  PRO-SWAT  procedure.  This 
was  due,  in  part,  to  workload  assessment  being  contingent 
upon  both  the  structure  of  the  materials  used  and  the 
relevant  characteristics  of  the  rater.  As  a  first  effort, 
however,  the  PRO-SWAT  technique  did  show  promise  as  a  tool 
to  predict  pilot  workload  without  having  to  construct  an 
operable  version  oi  a  proposed  system. 

RAM/ ACE  Replication 

Since  much  of  the  raw  data  from  the  original  RAM/ACE 
program  was  initially  unavailable,  an  attempt  was  made  to 
replicate  the  PRO-SWAT  portion  of  this  experiment.  This 
replication  effort,  under  the  author's  direction,  took 
place  in  1986,  approximately  four  years  after  the  initial 
PRO-SWAT  effort.  The  original  written  descriptive  material 
of  the  enhanced  F-15  cockpit  configuration  was  available 
and  was  used  with  only  minor  changes.  The  revised  version 
appears  in  Appendix  C  of  this  document,  which  contains  the 
entire  workload  prediction  survey  used  in  the  experiment 
replication.  After  completion  of  this  replication,  the 
original  PRO-SWAT  raw  data  was  found,  so  both  sets  of  data 
were  used  in  the  analysis  discussed  in  chapter  V. 


The  workload  prediction  survey  (Appendix  C)  was 
administered  to  eight  A-7  pilots  and  five  F-4  pilots.  Both 
sets  of  pilots  were  well -experienced  in  the  type  of  air-to- 
ground  mission  used  in  the  original  RAM/ACE  study.  The  A-7 
pilots  were  members  of  the  166  and  162  Tactical  Fighter 
Squadrons,  Ohio  Air  National  Guard  and  the  F-4  pilots  were 
members  of  the  89  Tactical  Fighter  Squadron,  U.S.  Air  Force 
Reserve.  The  A-7  is  a  single  seat  (pilot  only)  aircraft 
designed  for  the  air-to-ground  mission  only.  The  F-4 
carries  two  crt  /  members,  a  pilot  and  a  weapon  systems 
operator.  The  F-4  can  accomplish  both  air-to-ground  and 
air-to-air  missions.  Each  pilot  was  asked  to  answer  the 
survey  as  if  he  were  performing  duties  in  a  manner  he  was 
accustomed  to.  That  is,  A-7  pilots  answered  as  if  they 
were  the  only  crew  member  and  F-4  pilots  answered  as  if 
they  had  the  assistance  of  a  weapon  system  operator.  This 
is  the  same  manner  in  which  the  original  RAM/ACE 
measurements  were  taker. 

Even  though  both  sets  of  workload  estimates  were 
developed,  only  the  A-7  estimates  were  used  in  the 
remainder  of  this  research.  The  F-4  workload  estimates 
were  not  included  because  the  simulation  model  developed  to 
modify  workload  estimates  did  not  model  a  pilot  working 
with  another  crewmember.  This  is  a  much  more  complex 
situation  since  it  requires  consideration  of  the  sharing  of 
tasks  and  the  required  coordination  to  accurately  model 


such  a  situation.  In  general,  this  process  is  not  well 
understood,  so  no  attempt  was  made  to  include  it  in  the 
simulation  model.  The  intent  of  this  research  was  to 
explore  various  pilot  modeling  options.  Since  this  was  not 
possible  for  a  two-man  cockpit,  no  attempt  was  made  to 
include  F-4  data. 

The  workload  survey  (Appendix  C)  was  introduced  by  the 
author  to  the  pilots  in  groups  of  three  to  five  on  normal 
training  days.  All  pilot  subjects  were  volunteers  and 
received  no  compensation  for  their  participation  other  than 
their  normal  duty  pay.  After  a  short  introduction  on  the 
purpose  of  this  survey,  the  subjects  were  given  an  overview 
of  the  concept  of  workload  and  the  SWAT  measurement 
technique.  Demographic  data,  including  flying  experience, 
was  requested  and  recorded  for  each  subject.  This  was 
followed  by  a  SWAT  card  sort  by  each  subject.  A  copy  of 
the  SWAT  cards  and  accompanying  instructions  are  contained 
in  Appendix  A.  The  purpose  of  this  card  sort  is  to 
establish  a  linear,  calibrated  scale  on  which  each 
individual’s  SWAT  ratings  could  be  placed.  This  process  is 
described  in  detail  by  Reid,  Shingledecker,  and  Eggemeier 
(30).  The  results  of  the  SWAT  card  sorting  exercise  are 
contained  in  Appendix  B. 

After  all  pilots  completed  the  SWAT  card  sort  they 
were  given  a  copy  of  the  workload  survey  (Appendix  C) . 

They  were  asked  to  complete  this  survey  on  their  own  time 


and  mail  the  three  data  sheets  back  to  the  author. 

Ideally,  it  would  have  been  better  to  have  conducted 
this  survey  in  a  more  controlled  environment.  That  is, 
each  pilot  should  have  completed  this  survey  under  the 
supervision  of  the  author  so  that  questions  could  have  been 
answered  and  some  assurances  made  that  each  pilot  gave 
sufficient  time  and  attention  to  his  answers.  However, 
this  was  not  possible  due  to  time  constraints  set  by  each 
flying  squadron.  As  a  result,  each  pilot  was  simply  asked 
to  complete  the  survey  on  their  own  within  the  following 
two  weeks. 

All  pilots  involved  in  this  research  returned  their 
surveys.  After  receiving  all  these  surveys,  the  data  was 
reviewed  and  summarized.  These  summaries  were  returned  to 
each  pilot  along  with  copies  of  that  pilot's  original 
ratings.  Each  pilot  was  then  asked  to  compare  his  original 
ratings  to  the  summarized  ratings,  modify  his  own  ratings 
if  desired,  and  return  the  revised  ratings  to  the  author. 
The  purpose  of  this  review  and  revision  effort  was  to 
determine  if  knowledge  of  the  group's  ratings  would  affect 
the  individual  ratings  of  each  pilot.  It  was  planned  that, 
if  there  were  any  significant  changes  after  the  first 
review  and  revision  process,  the  process  would  be  repeated 
until  the  revisions  essentially  ceased,  that  is,  a  stable 
estimate  was  achieved.  The  final  set  of  revised  estimates 
would  become  the  set  of  data  for  the  1986  experiment  which 


would  be  compared  to  the  RAM/ACE  flight  simulator  data.  In 
this  research  only  one  iteration  was  required  since  none  of 
the  workload  estimates  was  changed  after  the  first 
iteration.  This  1986  data  would  also  be  used,  in 
conjunction  with  the  data  from  the  discrete  event 
simulation,  to  form  a  set  of  predictors  to  compare  with  the 
RAM/ACE  data.  The  details  of  this  are  described  in  the 
next  section. 

Experimental  Design  and  Data  Analysis 

The  general  purpose  of  this  research  is  to  determine 
if  the  use  of  a  simulation  model  of  a  pilot  can  improve 
workload  predictions  for  that  pilot.  To  accomplish  this 
purpose,  a  computer  simulation  model  was  built,  two  sets  of 
workload  predictions  were  made,  and  workload  measurement 
data  from  a  flight  simulator  exercise  was  used.  This 
section  describes  how  the  data  from  each  of  these  tasks  was 
used  to  draw  conclusions  about  the  value  of  using  a 
simulation  model  to  predict  workload. 

The  primary  output  of  the  computer  simulation  models 
was  three  metrics:  pilot  idle  time  (or  spare  time) ,  task 
interruption  rate,  and  simultaneous  task  rate.  All  three 
of  these  parameters  are  measured  as  a  per  cent  or, 
equivalently,  a  number  between  zero  and  one.  On  the  other 
hand,  the  primary  output  of  both  the  predictions  and  the 
SWAT  flight  simulator  workload  measurement  exercise  is  a 
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set  of  workload  metrics  in  three  dimensions  (Time  Load, 
Mental  Effort  Load,  and  Psychological  Stress  Load)  rated  as 
LOW,  MEDIUM,  or  HIGH.  Since  the  ultimate  goal  is  to  use 
all  of  this  information  together,  it  is  necessary  to 
develop  a  transformation  from  one  set  of  metrics  to  the 
other. 

Since  the  SWAT  methodology  was  used  throughout  this 
research,  the  decision  was  made  to  continue  with  that 
structure  and  transform  the  simulation  model  metrics  into 
the  SWAT  domain.  In  essence,  the  three  metrics,  derived 
from  the  computer  simulation  model,  become  surrogates  or 
replacements  for  the  Time  Load  dimension  of  the  SWAT 
methodology.  This  requires  that  a  method  be  found  to 
transform  the  three  metrics  (all  in  the  range  0.0  -  1.0)  to 
ratings  of  LOW,  MEDIUM,  and  HIGH.  The  method  used  to 
develop  this  transformation  was  to  ask  a  wide  variety  of 
people,  via  a  brief  survey,  to  estimate  these  metrics  for 
each  potential  SWAT  workload  rating  (L,  M,  H) .  The  survey 
and  the  detailed  results  are  presented  in  Appendix  G  and 
discussed  in  Chapter  V. 

After  completion  of  this  transformation,  all  required 
information  was  contained  entirely  within  the  SWAT 
structure  and  terminology.  At  this  point  it  was  possible 
to  construct  seven  candidate  predictors  for  evaluation  as 
model -augmented  improvements  to  the  totally  subjective  PRO¬ 
SWAT  predictions.  In  addition,  the  PRO-SWAT  ratings 
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themselves  were  used  as  an  eighth  predictor  and  as  a 
baseline  estimate  of  workload  by  which  to  compare  the  other 
four  predictors.  Five  of  these  candidate  predictors  were 
based  on  a  single-channel!  serial-processing  model  of  a 
pilot,  as  described  in  Chapter  IV.  These  five  candidate 
predictors  used  pilot  idle  time  and  four  different 
definitions  of  task  interruption  rate,  determined  from  the 
single  channel  pilot  model,  as  estimates  of  workload  in  the 
Time  Load  dimension.  The  remaining  two  candidate 
predictors  were  derived  from  the  multiple-channel,  parallel 
processing  model  of  a  pilot,  which  is  also  described  in 
Chapter  IV.  These  two  predictors  used  pilot  idle  time  and 
simultaneous  task  rate  as  estimates  of  workload  in  the  Time 
Load  dimension.  Figure  2  displays  the  structure  and 
relationships  of  these  terms  and  measures. 

By  using  pilot  idle  time  derived  from  both  the  single 
channel  model  and  the  multiple  channel  model,  conclusions 
can  be  drawn  about  the  appropriateness  of  each  model. 

Pilot  idle  time  or  spare  time  is  simply  a  measure  of  the 
proportion  of  time  the  pilot  is  not  occupied  performing  a 
required  task.  It  does  not  recognize  other  factors  which 
may  impact  in  the  Time  Load  dimension.  These  factors  would 
include  the  influence  of  tasks  interrupting  one-another  or 
the  influence  of  tasks  being  accomplished  simultaneously. 


Figure  2  Derivation  of  Time  Load  Predictors 


In  this  single-channel  model,  only  one  task  can  be 
accomplished  at  a  time,  with  higher  priority  tasks 
interrupting  lower  priority  tasks.  Therefore,  the  relative 
rate  at  which  this  happens,  the  task  interruption  rate, 
measures  the  effect  of  this  phenomena  on  the  Time  Load 
dimension.  A  single  channel  model  does  not  allow  for 
simultaneous  processing  of  more  than  one  task,  so  a 
simultaneous  task  rate  would  be  irrelevant  for  the  single 
channel  model  and  is,  in  fact,  always  equal  to  zero  for 
this  model. 

Although  it  appears  to  be  a  simple  concept,  it  is  not 
entirely  clear  what  constitutes  a  task  interruption,  or  how 
interruptions  should  be  measured.  One  possible  means  of 
quantifying  interruptions  is  to  count  only  the  number  of 
times  tasks  are  preempted  by  higher  priority  tasks  and 
normalize  by  dividing  by  the  total  number  of  task  arrivals. 
This  approach  would  yield  the  percent  of  tasks  interrupted 
by  preemption.  One  possible  problem  with  this  method  is 
that  low  priority  tasks  could  be  preempted  many  times, 
yielding  a  ratio  of  interrupts  to  arrivals  greater  than 
one.  This  value  is  not  intuitively  appealing  and  may 
present  problems  in  converting  task  interrupt  rates  to  SWAT 
workload  ratings  of  LOW,  MEDIUM,  and  HIGH.  This  problem 
can  be  overcome  by  increasing  the  number  of  arrivals  by  one 
each  time  a  preempted  task  re-enters  service. 


Another  possible  view  of  what  constitutes  a  task 
interruption  is  to  ignore  task  preemptions  and  simply  count 
the  number  of  times  a  task  arrives  while  the  pilot  is  busy 
with  another  task.  Perhaps  the  pilot's  recognition  of  the 
arrival  of  a  second  task  constitutes  an  interruption  and 
adds  to  the  pilot's  workload.  The  arrival  of  this  second 
task  could  involve  additional  workload  on  the  pilot's  part 
to  recognize  and  classify  the  task,  make  a  priority 
determination  for  the  two  tasks  involved,  and  either  effect 
the  preemption,  or  mentally  recognize  that  another  task 
awaits  his  attention  if  preemption  is  not  possible.  In 
this  case  task  interruption  rate  could  be  computed  by 
counting  the  number  of  times  that  tasks  arrive  while  the 
pilot  is  busy,  normalized  by  dividing  by  the  total  number 
of  arrivals.  Although  arrival  of  an  additional  task  could 
cause  workload,  no  special  provision  for  this  was  provided 
in  the  simulation  model. 

In  this  research,  each  of  these  possible 
interpretations  of  task  interruption  will  be  considered. 

The  relative  worth  of  each  interpretation  will  be 
determined  in  the  validation  process  of  comparing  PRO-SWAT 
predictions  to  SWAT  measurements. 

For  the  multiple  channel  model,  tasks  are  not  often 
interrupted,  but  additional  tasks  may  be  added  to  a  task 
currently  being  worked  and  processed  simultaneously.  By 
measuring  the  rate  at  which  tasks  are  completed 


simultaneously,  it  is  possible  to  capture  this  aspect  of 
the  Time  Load  dimension.  In  a  sense,  this  measure  goes 
beyond  the  concept  of  idleness  or,  conversely,  "busy-ness" . 
Simultaneous  task  rate  suggests  that  the  pilot  is  not  only 
busy,  but  is  occupied  with  a  given  number  of  parallel 
tasks,  on  the  average.  Although  some  tasks  may  be 
interrupted  in  a  multiple  channel  model,  the  effect  of 
these  interruptions  is  assumed  to  be  insignificant  compared 
to  the  number  of  tasks  completed  simultaneously. 

The  original  RAM/ACE  flight  simulator  exercise  was 
conducted  under  a  wide  variety  of  equipment  enhancements, 
mission  scenarios,  crew  size,  and  both  day  and  night 
conditions.  A  subset  of  these  conditions  was  selected  to 
provide  the  basis  for  comparing  the  five  predictors 
previously  mentioned.  Three  cockpit  equipment 
configurations  were  selected.  These  were  the  F-15C 
Advanced  Baseline  (ABL) ,  the  ABL  plus  Terrain 
Following/Terrain  Avoidance  (TF/TA) ,  and  the  ABL  plus 
Terrain/Threat  Avoidance  (TTA) .  This  eliminated  four  other 
possible  enhancements  from  consideration,  but  the  three 
selected  appeared  to  span  the  range  of  possible  pilot 
workload.  The  ABL  represents  the  enhancement  with  the 
fewest  automatic  features  and  suggests  the  possibility  of 
high  workload.  The  TF/TA  system  automates  the  problem  of 
avoiding  contact  with  the  ground,  somewhat  alleviating  the 
pilot's  workload.  The  TTA  system  automates  both  the 


terrain  avoidance  problem  and  the  hostile  threat  avoidance 
problem.  By  examining  all  three  of  these ,  situations  of 
high,  medium,  and  low  workload  were  be  examined, 
respectively. 

Data  was  available  for  both  day  and  night  missions  in 
the  RAM/ACE  study.  However,  to  reduce  the  work  involved, 
only  data  from  day  missions  was  used.  This  was  something 
of  an  arbitrary  decision.  Night-time  scenarios  could  have 
been  modeled  in  the  computer  simulation,  but  this  would 
have  required  modifying  some  of  the  input  data  since  it  was 
based  on  day-time  pilot  performance.  This  modification 
would  have  added  some  additional  uncertainty  since  this 
transformation  is  undefined.  RAM/ ACE  data  was  available 
for  both  one-man  and  two-man  crews,  but  only  the  data  for 
one-man  crews  was  used.  It  was  not  the  intent  of  this 
study  to  attempt  to  model  the  complex  interaction  process 
that  occurs  in  a  two-man  cockpit.  Several  mission 
scenarios  were  also  defined  for  each  equipment  enhancement. 
The  primary  difference  in  these  scenarios  was  the  final 
course  flown  to  the  target  and  the  relative  location  of 
turn  points  along  the  route  of  flight.  Since  it  was  felt 
that  different  inbound  courses  and  turn-points  approaching 
a  target  would  have  no  impact  on  workload,  these 
differences  in  scenarios  were  ignored.  That  is,  the  data 
for  different  scenarios  was  combined  and  treated  as  being 


homogeneous.  This  provided  more  RAM/ACE  data  for 
comparison  purposes. 

Summary 

This  chapter  describes  the  overall  model  developed  to 
predict  workload.  This  model  begins  with  the  SWAT 
methodology  and  incorporates  surrogate  Time  Load  predictors 
from  simulation  models.  These  predictors  are  changed  from 
the  values  of  the  computer  simulation  output  to  the  SWAT 
ratings  of  LOW,  MEDIUM,  and  HIGH  by  means  of  a 
transformation  developed  in  this  research. 

The  workload  predictions  of  this  research  are  compared 
to  the  workload  measurement  of  the  RAM/ ACE  study.  This 
study  is  discussed  and  a  replication  of  the  workload 
portion  of  that  study  is  also  discussed.  Finally,  the 
experimental  design  and  plan  for  data  analysis  is  also 
discussed.  The  details  of  the  actual  statistical  analysis 
are  discussed  in  Chapter  V.  For  the  interested  reader, 
details  of  the  computer  simulation  are  presented  in  the 
next  chapter  and  additional  information  is  found  in 
Appendix  E.  Listings  of  each  computer  model,  the  serial 
server  model  and  the  parallel  server  model,  can  be  found  in 
Appendix  F. 


CHAPTER  IV 


SIMULATION  MODEL 

This  chapter  describes  a  discrete-event  simulation 
model  developed  to  estimate  pilot  workload  in  the  Time  Load 
dimension  of  the  SWAT  methodology.  The  Time  Load  estimates 
derived  from  this  model  will  be  combined  with  subjective 
estimates  in  the  other  two  dimensions.  This  will  provide  a 
workload  predictor  which  will  be  compared  to  SWAT  ratings 
of  simulator  flights  using  the  same  scenarios.  Two  basic 
versions  of  this  simulation  model  were  developed.  The 
first  model  assumed  that  the  pilot  is  a  serial  processor, 
i.e.,  he  could  accomplish  only  one  task  at  a  time.  The 
second  version  allowed  parallel  processing  of  tasks,  but 
computed  a  service  time  penalty  when  more  than  one  task  was 
processed  at  a  time. 

Some  similarities  exist  in  both  versions.  In  each 
case,  the  basic  structure  of  a  queueing  model  is  used.  In 
this  model,  the  pilot  is  the  server  and  the  tasks  he 
accomplishes  are  his  customers.  Customers  are  served 
according  to  a  priority  system.  Higher  priority  tasks 
preempt  lover  priority  tasks,  within  a  priority  class,  a 


53 


first-come,  first-served  discipline  is  employed.  Three 
task  priority  classes  av  used:  operating  tasks, 
monitoring  tasks,  and  planning  tasks.  Operating  tasks  are 
those  required  for  aircraft  control  and  subsystem 
operation.  This  class  has  the  highest  priority  and  can 
tolerate  very  little  delay.  Monitoring  tasks  are  those 
which  require  the  pilot  to  scan  inside  and  outside  the 
cockpit  to  update  his  knowledge  of  the  aircraft's  status 
and  situation.  This  second  class  can  accept  short  delays 
without  serious  consequences.  The  lowest  priority  class, 
planning  tasks,  can  be  deferred  into  idle  periods  for 
completion.  Some  tasks,  such  as  aircraft  control,  are 
recurrent  and  are  modeled  as  repeat  customers,  other 
tasks,  such  as  passing  a  navigation  turn  point,  occur  only 
at  specified  points  in  a  flight  and  are  modeled  as  non¬ 
repeating  customers. 

This  simulation  model  is  written  in  SIMSCRIPT  II. 5  and 
was  run  on  both  an  IBM  3081  and  a  VAX  11/705.  Run  times  on 
the  IBM  were  approximately  15  CPU  seconds  for  the  serial 
processing  model  and  21  CPU  seconds  for  the  parallel 
processing  model.  On  the  VAX,  the  times  were  14  CPU 
seconds  and  41  CPU  seconds,  respectively.  The  model 
consists  of  twelve  separate  event  routines  which  create 
tasks  to  be  accomplished  by  the  pilot.  Of  these  twelve 
discrete  event  routines,  seven  are  repetitive  while  the 
remainder  occur  only  once.  The  inter-arrival  time  for  each 
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repetitive  task  is  random  and  is  determined  by  Monte  Carlo 
sampling  from  predetermined  probability  distributions.  The 
time  required  for  the  pilot  to  complete  each  task  is  also 
random  and  is  determined  by  Monte  Carlo  sampling  from 
another  appropriate  probability  distribution.  The  specific 
tasks,  distributions,  and  parameters  used  in  this  model  are 
documented  in  Appendix  E.  Other  routines  also  exist  to 
accomplish  specific  computations  or  to  print  intermediate 
and  final  results.  Figure  3  shows  the  generic  network 
structure  employed  in  both  the  single  channel  and  parallel 
proces.  ng  models. 

Single  Channel  Model 

This  version  of  the  model  assumes  that  a  pilot 
processes  all  tasks  sequentially.  That  is,  he  can  process 
only  one  task  at  a  time  and  cannot  begin  a  second  task 
until  the  first  is  completed  or  preempted.  Any  task  which 
arrives  while  the  pilot  is  working  on  another  task  must 
have  a  higher  priority  for  service  to  preempt  the  working 
task.  If  it  is  unable  to  preempt,  the  newly  arrived  task 
must  wait  until  the  old  task  is  completed  and  the  pilot  is 
free  to  begin  service.  All  tasks  waiting  for  service  are 
ranked  by  priority.  Once  a  task  has  been  preempted,  it 
returns  to  the  waiting  line  to  await  another  chance  for 
service.  Preempted  tasks  do  not  use  their  original  service 
time  for  subsequent  service,  but  resample  to  determine  a 


new  service  time.  Resampling  is  done  to  reflect  the 
changed  aircraft  state  when  the  task  re-enters  service.  It 
is  assumed  that  previously  accomplished  portions  of  tasks 
have  to  be  repeated,  changed,  or  corrected. 

The  primary  estimators  of  Time  Load  produced  by  the 
single  channel  model  are  pilot  idle  time  and  the  ratio  of 
interrupted  tasks  to  task  arrivals.  These  estimators  are 
described  in  Chapter  III  and  are  used  to  predict  pilot 
workload.  Figures  4  and  5  display  the  task  arrival  and 
service  completion  processes  respectively  for  the  single 
server  model. 


Parallel  Processing  Model 

The  second  version  of  this  model  assumes  that  a  pilot 
can  process  more  than  one  task  at  a  time.  This  is 
accomplished  by  processing  tasks  in  parallel.  The  number 
of  parallel  channels  available  is  not  fixed,  but  is 
dependent  on  the  pilot  resources  required  by  the  tasks 
demanding  service,  wickens*  (46)  multiple  resource  model 
is  used  to  define  available  pilot  resources. 

When  a  new  task  arrives  and  the  pilot  is  idle,  the 
pilot  will  immediately  begin  work  on  that  task.  If  the 
pilot  is  not  idle,  but  sufficient  pilot  resources  are 
available,  the  pilot  will  also  begin  work  on  the  second 
task  immediately.  If  resources  are  not  available,  the 
model  will  preempt  a  lower-priority  task,  if  possible.  If 


Figure  4  Task  Arrival  Process ,  Single  Server  Model 
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Figure  5  Service  Completion  Process,  Single  Server  Model 
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preemption  is  not  possible,  the  new  task  waits  until 
resources  become  available  so  that  the  pilot  can  begin  work 
on  this  new  task. 

Within  the  simulation,  the  availability  of  resources 
is  determined  in  the  following  manner.  A  2  x  3  matrix 
maintains  a  listing  of  resources  currently  required  by 
tasks  on  which  the  pilot  is  working.  This  matrix  is  shown 
in  Table  1.  All  values  in  this  matrix  are  initially  set 
equal  to  zero.  When  the  first  task  arrives,  the 

Table  1  Multiple  Resource  Model  Matrix. 


MODALITY 

CODE 

STAGE 

Auditory 

Visual 

Spatial 

Verbal 

Responding 

Encode/  antral 

Processing 

resource  requirements  of  that  task  are  used  to  change 
corresponding  matrix  cell  values  to  equal  one,  while  the 
others  remain  equal  to  zero.  If  a  second  task  arrives 
prior  to  the  completion  of  the  first,  the  resource 
requirements  of  the  new  task  are  temporarily  added  to  those 
of  the  old  task.  If  this  results  in  no  more  than  two 
resources  being  used  simultaneously,  the  second  task  is 
assigned  to  the  pilot  to  work.  If  three  resources  are 
required  simultaneously,  a  further  computation  is  required 


to  determine  if  both  tasks  can  be  accomplished 
simultaneously.  This  computation  is  the  determination  of  a 
Degradation  Factor.  It  is  based  on  the  number  of 
overlapping  task  resource  requirements.  The  scheme  used  to 
determine  this  is  shown  in  Table  2.  The  values  in  this 
table  are  derived  from  the  conflict  matrix  developed  by 
North  (23) .  This  Degradation  Factor  is  used  both  for 
determining  if  an  additional  task  can  be  assigned  to  the 
pilot  and  computing  the  amount  of  service  time  degradation 
resulting  from  that  assignment. 

Table  2  Degradation  Factors. 


NUMBER  OF 

OVERLAPPING  RESOURCES 

DEGRADATION 

FACTOR 

0  (With  2  tasks) 

Uniform  (0.1,  0.3) 

1 

Uniform  (0.3,  0.6) 

2 

Uniform  (0.6,  0.9) 

3 

Uniform  (0.9,  1.1) 

4  (Or  more) 

1.1 

Whenever  a  new  task  is  assigned  to  the  pi lot , 
resulting  in  the  pilot  working  on  more  than  one  task 
(parallel  processing) ,  the  service  times  for  all  tasks 
being  worked  are  increased.  This  reflects  the  performance 
degradation  resulting  from  parallel  processing,  as  compared 


to  serial  processing  [Norman  &  Bobrow  (22)  and  Wickens, 
Mountford,  &  Schreiner  (44)].  Service  times  are  increased 
by  computing  a  Degradation  Factor.  This  factor,  normally 
between  0  and  1,  is  multiplied  by  the  single-task  service 
time  and  the  product  added  to  that  service  time.  For 
example,  if  a  degradation  factor  of  0.5  were  computed,  all 
affected  task  service  times  would  be  multiplied  by  a  factor 
of  1.5,  or  effectively  increased  by  50%.  If  preemption  is 
not  possible  and  the  Degradation  Factor  exceeds  1.0,  then 
the  new  task  would  be  assigned  to  wait  until  resources  are 
available.  A  Degradation  Factor  limit  of  1.0  is  a 
subjective,  arbitrary  limit.  It  represents  doubling  a 
service  time  and  assumes  that  it  is  better  to  defer  a  task 
to  an  idle  period  than  to  double  the  current  service  times 
of  all  tasks  being  worked  at  that  time.  Each  time  the 
pilot  begins  work  on  a  new  task,  the  resources  being  used 
to  work  on  all  tasks  are  updated.  Similarly,  when  a  task 
is  preempted  or  completed,  resource  status  is  also  updated. 
Figures  6  and  7  display  the  task  arrival  and  service 
completion  processes  respectively  for  the  parallel 
processing  model. 

The  method  used  to  degrade  service  times  is  based  on 
simply  counting  the  number  of  required  resource  "overlaps** 
and  computing  a  Degradation  Factor  based  on  this  count. 

This  procedure  is  a  modeling  simplification  since  it 
ignores  existing  evidence  [Norman  and  Bobrow  (22)  and 
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Figure  6  Task  Arrival  Process,  Parallel  Processing  Model 
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Figure  7 


Service  Completion  Process,  Parallel  Processing 
Model 


wickens,  Mountford,  and  Schreiner  (44)]  that  the  type  of 
resource  overlap  is  important  in  determining  the  actual 
degradation  of  service  times  for  dual  task  performance  over 
single  task  performance.  Very  little  data  exists  to 
accurately  determine  these  degradations  for  specific  task 
combinations  if  resource  type  were  considered.  However, 
the  degradations  used  were  based  on  North's  work  (24)  and 
agree  with  the  degradation  factors  he  presented. 

Therefore,  since  data  for  specific  resource  overlaps  was 
not  available,  an  assumption  was  made  that  simply  counting 
the  number  of  required  resource  overlaps  was  a  reasonable 
modeling  approach. 

Task  Inter-Arrival  and  Service  Time  Development 
All  times  were  first  determined  for  the  ABL 
configuration  in  the  Penetration  Segment  and  then  adjusted 
for  other  segments  and  configurations.  These  are  presented 
in  Table  30.  For  the  ABL,  service  times  in  the  Penetration 
Segment  are  based  on  exp  rimental  results  or  handbook  data. 
The  service  rate  for  flight  control  inputs  involved  in  the 
FLY  AIRCRAFT  task  are  based  on  a  flight  simulator 
experiment  done  by  Groves  and  Kaercher  (11).  Service  times 
are  assumed  to  be  distributed  exponentially  since  the  mean 
and  standard  deviation  determined  experimentally  (1.509  and 
1.414)  are  approximately  equal  and  the  cumulative 
distribution  function  presented  by  Groves  and  Kaercher 
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strongly  resembles  an  exponential  distribution.  However, 
insufficient  data  is  presented  by  Groves  and  Kaercher  to 
determine  the  statistical  confidence  in  this  assumption. 

All  other  tasks  in  the  Penetration  Segment  derive 
their  service  times  from  standard-  values  as  described  by 
Siegel  and  Wolf  (39).  These  service  times  are  assumed  to 
be  normally  distributed.  This  seems  to  be  a  logical 
assumption  since  the  tasks  are  procedurally  oriented  and 
should  require  approximately  the  same  amount  of  time  to 
accomplish  on  each  repetition. 

Inter-arrival  times  for  the  FLY  AIRCRAFT  task  are 
again  derived  from  experimental  results  by  Groves  and 
Kaercher.  These  authors  assumed  this  data  to  be  normally 
distributed,  although  they  do  not  provide  sufficient  data 
to  verify  this.  Their  assumption  vas  used  in  this  research 
since  it  worked  well  in  their  research.  All  other  inter¬ 
arrival  times  are  estimates  based  on  the  author's 
experience.  In  addition,  these  estimates  compared 
favorably  to  those  of  Groves  and  Kaercher  (11).  Those 
tasks  labeled  with  an  "SR"  in  Table  31  are  repetitive  and 
are  assumed  to  be  distributed  normally.  Tasks  labeled  "N* 
are  Ron-repetitive  and  occur  only  once. 

The  mean  inter-arrival  and  service  times  of  tasks 
accomplished  during  the  Target  Acquisition  Segment  are 
derived  from  those  of  the  Penetration  Segment.  All  service 
times  remain  unchanged,  but  the  mean  of  seven  of  the  twelve 


inter-arrival  times  are  changed.  One  of  these,  the  FLY 
AIRCRAFT  task,  is  increased  slightly  to  recognize  the 
decreased  rate  of  flight  control  input  required  of  the 
pilot  while  executing  a  H POP-UP"  maneuver  rather  than 
continuing  to  fly  a  terrain  following  profile.  The  other 
six  tasks  (NAVIGATE,  ASSESS  ENVIRONMENT,  ASSESS  MISSION 
PLAN,  MONITOR/ ASSESS  AIRCRAFT  SYSTEMS,  CONFIGURE  AIRCRAFT 
SYSTEMS,  and  DEFENSIVE  MANEUVER)  all  have  decreased  inter¬ 
arrival  times  to  reflect  increased  activity  in  these  areas. 
A  pilot's  primary  concern  in  this  phase  is  to  detect  and 
identify  the  target.  Precise  altitude,  heading,  or 
airspeed  control  is  not  required. 

The  same  philosophy  is  used  in  determining  parameters 
for  the  tasks  of  the  Weapon  Delivery  Segment.  The  mean  of 
the  FLY  AIRCRAFT  inter-arrival  time  distribution  is  the 
same  as  that  used  during  the  terrain-following  Penetration 
Segment  since  precise  heading,  airspeed,  and  altitude  are 
required  for  accurate  weapon  release.  All  other  task 
parameters  remain  unchanged  from  the  Target  Acquisition 
segment . 

The  inter-arrival  and  service  time  distributions 
developed  for  the  ABL  configuration  were  used  as  a  basis 
for  both  the  TF/TA  and  TTA  configurations.  The  fundamental 
difference  between  the  TF/TA  and  ABL  configurations  is  that 
the  former  automatically  controls  pitch  and  bank  to  provide 
for  terrain  following  and  terrain  avoidance.  The  latter 
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only  provides  commands  for  terrain  following  which  require 
a  manual  pilot  response.  This  difference  is  accounted  for 
by  increasing  the  mean  inter-arrival  time  for  the  FLY 
AIRCRAFT  task  in  the  Penetration  Segment  for  the  TF/TA 
configuration.  The  amount  of  increase  is  based  on  the 
author’s  judgment  and  reflects  the  possibility  that,  even 
though  TF/TA  is  fully  automatic,  the  pilot  may  manually 
override  the  system  periodically. 

All  service  and  inter-arrival  distribution  parameters 
for  the  Target  Acquisition  Segment,  with  the  TF/TA 
configuration,  are  identical  to  the  ABL  configuration.  In 
this  segment,  terrain-following  and  avoidance  are  not  being 
accomplished  so  the  benefits  of  the  automatic  system  are 
not  obtained.  In  the  Weapon  Delivery  Segment  for  the  TF/TA 
configuration,  the  pilot  is  again  using  TF/TA  to  maintain 
terrain  separation.  However,  now  he  must  integrate  this 
information  with  information  related  to  his  target 
location.  Therefore,  he  will  require  additional  manual 
inputs  compared  to  the  Penetration  Segment,  but  not  as 
frequently  as  would  be  required  by  the  ABL  configuration. 
All  other  parameters  remain  unchanged  from  the  ABL 
configuration. 

In  addition  to  providing  automatic  terrain  avoidance, 
the  TTA  configuration  also  provides  automatic  threat 
avoidance.  This  impacts  the  inter-arrival  time 
distribution  parameters  in  three  ways.  First,  the  inter- 


69 

arrival  mean  for  the  FLY  AIRCRAFT  task,  for  the  Penetration 
Segment,  is  equal  to  the  mean  for  the  TF/TA  configuration. 
This  reflects  the  fact  that  the  same  automatic  terrain 
avoidance  is  available,  so  the  load  on  the  pilot  from  this 
task  is  the  same  as  the  TF/TA  configuration.  Second,  since 
automatic  threat  avoidance  is  provided,  the  frequency  of  a 
DEFENSIVE  MANEUVER  task  is  decreased.  Therefore,  the  mean 
inter-arrival  time  for  this  task  is  assumed  to  double. 
Third,  this  configuration  requires  that  two  different 
displays  be  monitored,  the  Head-Down  Display  and  the  Threat 
Display.  This  requires  that  the  mean  inter-arrival  time 
for  the  MONITOR/ASSESS  AIRCRAFT  SYSTEMS  task  be  decreased 
to  reflect  the  increase  in  pilot  activity. 

In  the  Target  Acquisition  Segment,  the  enhanced 
features  of  the  TTA  configuration  provide  no  benefit  to  the 
pilot.  As  a  result,  the  inter-arrival  and  service  rate 
parameters  are  the  same  as  those  used  for  the  ABL  and  TF/TA 
configurations . 

For  the  Weapon  Delivery  Segment,  the  inter-arrival 
time  distribution  parameters  for  the  TTA  configuration 
differ  from  the  TF/TA  configuration  in  three  ways.  First, 
monitoring  two  displays  for  the  TTA  configuration  requires 
a  decreased  mean  inter-arrival  time  for  the  MONITOR/ASSESS 
AIRCRAFT  SYSTEMS  task  compared  to  the  TF/TA  configuration. 
Second,  the  mean  inter-arrival  time  for  the  DEFENSIVE 
MANEUVER  task  was  increased  due  to  the  automatic  features 
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of  the  TTA  system.  Finally,  the  mean  inter-arrival  time 
for  the  FLY  AIRCRAFT  task  was  decreased  due  to  the  increase 
in  control  inputs  required  in  this  segment. 

Parallel  Server  Model  Calibration 

The  previous  section  described  how  the  single  server 
model  was  developed.  Once  this  was  complete,  it  was 
necessary  to  calibrate  the  parallel  server  model  to  assure 
that  at  some  point  it  matched  the  performance  of  the  single 
server  model.  One  of  the  goals  of  this  research  was  to 
compare  the  performance  of  a  single  server  model  and  a 
parallel  server  model.  To  accomplish  this  goal,  it  was 
necessary  to  provide  a  common  starting  point  for  both 
models  where  the  output  performance  measures  were  as 
closely  matched  as  possible.  Once  this  baseline  was 
established,  any  subsequent  disparity  in  performance  would 
be  attributable  to  the  model,  not  the  starting  point. 

Since  the  input  parameters  for  the  single  server  model 
had  already  been  determined,  the  parallel  server  model  was 
calibrated  to  it  for  the  ABL  configuration.  This  was  done 
by  adjusting  the  mean  service  rates  for  the  parallel  server 
model  to  find  the  point  which  approximately  resulted  in  the 
same  utilization  rate  as  the  single  server  model.  For  this 
model,  changes  to  the  service  rate  were  examined  so  that 
the  same  task  arrival  rates  used  in  the  single  server  model 
were  also  employed  in  the  parallel  processing  model.  The 
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results  of  these  adjustments  in  service  rates  are  shown  in 
Figure  8.  In  this  figure,  the  solid  line  represents  the 
parallel  model  performance  as  a  function  of  service  rate 
changes.  The  dashed  line  is  the  idle  time  ratio  for  the 
serial  model.  The  point  at  which  the  lines  cross  represent 
the  service  rate  at  which  the  idle  time  ratio  from  both 
models  was  equal. 

From  Figure  8  it  was  apparent  that  no  single  service 
rate  yielded  identical  performance  between  the  single 
server  model  and  parallel  server  model.  Therefore,  it  was 
noted  that  the  total  minimum  squared  error  for  all  three 
flight  phases  occurred  at  an  increase  of  2%  in  service 
rates,  as  shown  in  Table  3.  However,  the  mean  squared 
error  at  a  0%  change  in  service  rate  was  only  slightly 
higher,  so  the  parallel  server  model  was  evaluated  with  the 
same  arrival  rates  and  service  rates  as  used  in  the  single 
server  model. 

Model  Verification 

Both  versions  of  this  model  are  relatively  complex  in 
their  final  versions.  Because  of  this  complexity,  it  is 
difficult  to  verify  that  each  version  is  an  accurate 
representation  of  a  queueing  system  model.  To  overcome 
this  problem,  steps  were  taken  during  model  development  to 
verify  the  proper  functioning  of  each  version,  as  it  was 
developed. 
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Figure  8  Parallel  Server  Model  Calibration 


Table  3  Utilization  Rate  Squared  Error:  Parallel 
Server  Model 


SERVICE  RATE 

SQUARED  DIFFERENCES 

CHANGES 

PEN 

ACQ 

WPN 

TOTAL 

0  % 

.00238 

.00203 

.00000 

.00441 

1  % 

.00174 

.00265 

.00000 

.00439 

2  % 

.00120 

.00314 

.00002 

.00436 

3  % 

.00071 

.00360 

.00005 

.00437 

4  % 

.00039 

.00397 

.00006 

.00442 

5  % 

.00016 

.00436 

.00008 

.00460 

For  the  single  channel  model,  an  initial  version  was 
created  with  a  single  task  source  and  exponentially 
distributed  inter-arrival  and  service  times.  Using  this 
simplified  model,  it  was  possible  to  compare  steady-state 
output  parameters  (mean  waiting  time,  mean  system  time, 
etc.)  for  the  simulation  model  with  known  analytical, 
steady-state  results.  This  essentially  verified  that  the 
arrival  and  servicing  of  customers  were  handled  properly 
and  output  statistics  were  computed  properly. 

Once  this  was  verified,  it  was  possible  to  add 
additional  tasks  and  a  priority  system.  Giffin  (9)  shows 
that  analytic  results  are  available  for  a  preemptive 
priority  system  with  a  single  server  and  any  number  of 
priority  classes,  each  having  a  common  mean  service  rate. 
Using  his  formulation,  it  was  also  possible  to  compare 
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simulation  results  with  analytic,  steady-state  results. 
These  results  compared  favorably  and  indicated  that  the 
simulation  model  describing  a  multiple-task  priority 
system,  in  a  simplified  form,  was  accurate.  The  change 
from  this  form  to  the  final  form  was  relatively  simple  in 
that  it  required  modifying  only  the  appropriate  rates  and 
distributions,  not  the  structure  of  the  model.  This 
procedure  lends  confidence  that  the  single  channel  model 
reasonably  represents  one  version  of  a  pilot  accomplishing 
tasks  in  a  cockpit  environment. 

Verification  of  the  parallel  processing  model  was 
accomplished  in  a  similar  manner.  A  simplified  version  of 
the  model  was  developed  which  allowed  parallel  processing 
in  two  channels  simultaneously.  A  single  customer  source 
was  provided  and  the  arrival  rate  and  service  rates  were 
selected  to  load  the  system  at  a  0.75  utilization  rate. 
Identical  service  rates  were  used  in  each  parallel  channel. 
Steady-state  simulation  results  were  compared  to  analytic 
results.  The  procedure  was  then  repeated  with  three  and 
four  channels.  These  comparisons  showed  that  the  model  was 
accurately  representing  a  multiple-channel  system. 

Verification  of  this  model  by  comparing  steady-state 
simulation  results  to  analytically-derived  values  for  any 
added  complexity  was  virtually  impossible  since  analysis 
was  not  possible  for  more  complex  systems.  Therefore,  the 
following  procedure  was  used  to  give  some  confidence  in  the 
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final  model.  Starting  with  the  simplified  version  of  the 
parallel  processing  model  previously  described,  several 
features  were  added.  These  included  a  priority  system, 
degraded  and  upgraded  service  rates,  and  a  logic  system  to 
determine  if  an  additional  channel  would  be  available  to 
service  another  task  in  parallel.  These  features  were 
added  one  at  a  time  and  resulting  performance  parameters 
closely  monitored  to  assure  that  they  were  reasonable.  For 
example,  when  the  priority  system  was  first  tested,  the 
average  system  time  for  each  class  of  customers  was 
computed.  Logically,  a  higher  priority  class  should  have  a 
lower  average  system  time  than  a  lower  priority  class. 

This  result  was  verified,  as  well  as  other,  similar  logical 
conclusions. 

When  the  final  version  of  the  parallel  processing 
simulation  model  was  completed,  several  runs  were  made 
which  provided  a  trace  of  each  customer  arrival  and  service 
event  completion.  This  allowed  a  manual  reconstruction  of 
the  events  and  statistics  of  each  run  to  be  made.  While 
this  was  tedious,  it  verified  that  the  simulation  of  the 
parallel  processing  model  was  performing  as  expected  and 
could  be  used  with  some  degree  of  confidence. 

Model  Output 

Both  versions  of  this  simulation  model  produce 
estimates  of  pilot  idle  time.  In  this  context,  idle  time 
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is  defined  as  the  ratio  of  time  spent  not  working  on  any 
task  to  total  time.  In  the  single  channel  version ,  it  is  a 
simple  matter  to  compute  idle  time;  the  pilot  is  either 
working  on  a  task  (busy)  or  not  working  on  a  task  (idle). 
The  same  concept  is  used  in  the  multiple  resource  model. 

The  pilot  is  idle  if  he  is  not  working  on  any  task.  He  is 
considered  busy  if  he  is  working  on  at  least  one  task.  The 
number  of  tasks  being  worked  does  not  influence  idle-time 
computations  for  this  version.  There  is  no  such  thing  as 
degrees  of  ’'busy-ness" ;  the  pilot  is  either  busy  or  idle. 

If  a  pilot  is  working  on  multiple  tasks  simultaneously  his 
workload  will  be  greater  than  if  he  is  only  working  on  a 
single  task.  Idle  Time  is  not  intended  to  measure  this, 
rather  an  alternative  measure.  Simultaneous  Task  Rate,  will 
be  used  to  account  for  multiple  tasks  being  completed 
simultaneously. 

The  single  channel  model  also  produces  another  Time 
Load  estimator,  the  ratio  of  task  interruptions  to  total 
arrivals.  Several  possible  definitions  and  interpretations 
of  a  task  interruption  are  possible.  One  possible 
interpretation  would  be  to  count  an  interruption  only  when 
one  task  is  preempted  by  another.  Another  view  might  be 
that  an  interruption  occurs  anytime  a  new  task  arrives 
while  the  pilot  is  working  on  another  task,  regardless  of 
whether  a  preemption  occurs.  This  suggests  that  the 
pilot1 s  simple  recognition  of  a  task  arriving  constitutes 


an  interruption.  Since  it  is  desired  that  any  task 
interruption  measure  be  normalized  between  0  and  1,  a  task 
interruption  measure  can  be  computed  by  dividing  total 
interruptions  by  total  task  arrivals,  for  either 
definition.  However,  total  task  arrivals  is  open  to 
interpretation.  It  is  obvious  that  the  original  arrival  of 
each  task  should  be  counted,  but  should  subsequent  arrivals 
of  previously  preempted  tasks  be  counted?  Since  there  was 
no  obvious  answer  to  this,  it  was  decided  to  consider  both 
counting  and  not  counting  preempted  tasks  as  arrivals. 

This  led  to  a  total  of  four  possible  definitions  of 
task  interruption  rate,  as  shown  in  Table  4. 

The  parallel  processing  model  also  determines  the 
average  number  of  tasks  being  worked  by  the  pilot.  This  is 
used  as  an  estimator  of  Time  Load.  In  this  version,  the 
pilot  can  work  on  more  than  one  task  at  a  time  so  the 
number  of  task  interruptions  is  not  a  pertinent  measure  of 
Time  Load.  Instead,  the  average  number  of  tasks  being 
worked  on  simultaneously  is  a  more  accurate  measure  of  Time 
Load.  In  the  model,  this  is  computed  by  maintaining  a 
time-weighted  average  of  the  number  of  tasks  being  worked 
by  the  pilot.  The  method  used  to  combine  these  estimators 
of  Time  Load  with  other  subjective  estimators  was  presented 
in  Chapter  III. 
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Table  4  Task  Interruption  Predictor  Definitions 


Predictor 

Interruption 

Defined  By: 

Subsequent  Arrival 

of  Preempted  Tasks: 

A 

Preemption 

Counted 

B 

Preemption 

Not  Counted 

C 

Arrival  of 

Counted 

New  Task 

D 

Arrival  of 

Not  Counted 

New  Task 

Summary 

This  chapter  described  two  discrete  event  simulation 
models  developed  to  provide  surrogate  predictors  for  the 
Time  Load  dimension  of  the  SWAT  methodology.  One  of  these 
models  assumed  tasks  were  completed  serially  and  the  other 
model  assumed  that  tasks  could  be  processed  in  parallel. 

The  number  of  tasks  processed  in  parallel  was  determined  by 
embedding  wickens'  multiple  resource  model  in  the 
simulation  model. 

Also  described  in  this  chapter  is  the  development  of 
task  inter-arrival  and  service  time  distributions.  Further 
details  can  be  found  in  Appendix  E.  Task  inter-arrival 
times  and  service  times  were  first  developed  for  the  serial 
model,  inter-arrival  times  were  identical  for  the  parallel 
Service  times  for  the  parallel  model  were 


model. 
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calibrated  to  the  serial  model  to  replicate  performance  for 
the  ABL  configuration. 

Finally,  the  process  used  to  verify  these  models  was 
discussed.  Model  validation  will  be  considered  in  the 
overall  process  of  validating  the  entire  workload 
prediction  model.  The  next  chapter  describes  the  results 
produced  by  these  models  and  leads  into  a  discussion  of  the 
workload  prediction  model  validation. 


CHAPTER  V 


RESULTS 


Introduction 

Three  basic  sets  of  workload  data  were  used  in  this 
study.  One  of  these  was  the  workload  ratings  derived  from 
the  flight  simulator  experiment  of  the  RAM/ACE  program. 

The  other  two  sets  were  workload  predictions,  one  derived 
as  part  of  the  RAN/ ACE  study  and  the  other  accomplished 
during  this  research.  The  value  of  these  predictions  can 
only  be  determined  by  comparing  them  to  the  RAH/ACE 
workload  ratings.  A  complete  listing  of  the  basic 
predictions  and  ratings  is  found  in  Appendix  X.  For 
comparison  purposes,  the  mean  values  of  these  observations 
are  shown  in  Table  5.  SWAT  workload  ratings  are  scaled 
between  o  and  100,  where  0  represents  no  workload  and  100 
represents  the  highest  possible  level  of  workload.  Other, 
intermediate  workload  levels  are  scales  between  0  and  100 
by  the  SWAT  methodology  (29). 

In  addition  to  comparing  the  basic  predictions  to  the 
workload  ratings,  other  predictions  were  developed  and 
compared  also.  These  additional  predictions  were  developed 
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Table  5  Workload  Ratings  and  Predictions  Summary 
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METHOD 

CONFIGURATION 

FLIGHT  PHASE 

PEN 

ACQ 

WPN 

SWAT 

ABL 

31.5 

63.1 

68.1 

RATINGS 

TF/TA 

28.3 

40.1 

65.4 

TTA 

21.4 

36.1 

57.3 

PRO-SWAT 

ABL 

46.8 

52.5 

82.3 

(1982) 

TF/TA 

29.0 

38.9 

41.3 

PREDICTIONS 

TTA 

22.1 

32.8 

57.8 

PRO-SWAT 

ABL 

51.7 

56.6 

50.1 

(1986) 

TF/TA 

54.2 

38.3 

35.2 

PREDICTIONS 

TTA 

39.9 

42.8 

39.3 

by  modifying  both  basic  sets  of  predictions  with  output 
from  the  computer  simulation  described  in  Chapter  IV.  This 
resulted  in  fourteen  additional  predictor  sets, 
ultimately,  each  of  these  predictor  sets  was  compared  to 
the  workload  ratings  to  evaluate  the  accuracy  of  each 
predictor.  This  chapter  describes  that  process  and  details 
the  results  of  the  comparison.  These  results  lead  to  the 
conclusions  drawn  in  the  next  chapter  concerning  the 
accuracy  of  the  predictors.  Since  some  assumptions  were 
made  in  accomplishing  this  research,  the  sensitivity  of  the 
results  to  these  assumptions  is  examined  in  the  final 
portion  of  this  chapter. 


Computer  Simulation  Results 

As  described  in  Chapter  IV,  two  basic  simulation 
models  were  built  to  augment  the  workload  predictions.  One 
model  assumed  the  pilot  could  accomplish  only  one  task  at  a 
time  while  the  second  model  assumed  the  pilot  could  process 
more  than  one  task  in  parallel.  The  former  model  estimated 
pilot  idle  time  and  task  interruption  rate  as  surrogates 
for  the  Time  Load  Dimension  of  workload.  The  latter  model 
developed  estimates  for  pilot  idle  time  and  simultaneous 
task  rate  as  workload  surrogates.  Each  model  was  run  for 
25  replications  to  more  accurately  estimate  mean  values  and 
to  determine  the  variance  of  the  estimators. 

The  results  of  these  computer  runs  are  summarized  in 
Tables  6  and  7.  These  results  show  the  mean  values 
computed  over  the  25  replications  as  well  as  the  standard 
error.  In  all  cases,  the  error  deviation  was  considered  to 
be  small  enough  to  disregard  the  use  of  variance  reduction 
techniques.  This  small  standard  error  also  suggested  that 
it  was  unnecessary  to  make  additional  runs  beyond  25.  As 
will  be  shown  in  the  next  section,  it  will  be  necessary  to 
convert  these  moan  predictor  values  to  ratings  of  1,  2,  or 
i  (LOW,  MEDIUM,  or  HIGH)  to  use  with  the  SWAT  methodology. 
The  relatively  low  standard  error  shown  in  these  tables 
generally  has  no  effect  on  this  conversion  process. 


Table  6  Predictor  Values,  Single  Server  Model 


PREDIC-  CONFIG- 
TOR  URATION 


SS-PIR 


SS-TI(A) 


SS-TI(B) 


SS-TI(C) 


SS-TI(D) 


FLIGHT  PHASE 


PEN  ACQ 


MEAN  S.E.  |  I  S.E. 


.389 

.609 

.588 


ABL 

TF/TA 

TTA 


ABL 

TF/TA 

TTA 


.131 

.070 

.072 

.152 

.077 

.079 

.538 

.309 

.297 

.467 

.287 

.321 


.004 

.006 

.006 

.006 

.007 

.007 


.008 

.008 

.010 


.141 

.130 

.146 

.177 
.3  59 
.180 

.738 

.528 

.476 

.631 

.453 

.566 


.017 

.016 

.015 

.024 

.021 

.021 

.025 

.031 

.025 

.022 

.022 

.034 


WPN 


MEAN  S 


.037  .010 

.124  .017 

.124  .019 


.011 

.015 

.016 

.014 

.021 

.021 

.016 

.029 

.028 

.015 

.025 

.032 


.100 

.142 

.139 

.944 

.813 

.705 

.861 

.714 

.801 


Table  7  Predictor  Values.,  Parallel  Server  Model 


FLIGHT  PHASE 


PREDIC¬ 

TOR 


PS-PIR 


PS-STR 


CONFIG¬ 

URATION 


ABL 

TF/TA 

TTA 

ABL 

TF/TA 

TTA 


.570  .007 

.677  .004 

.663  .004 

.352  .009 

.255  .009 

,241  .007 


.324  .020 

.449  .018 

.440  .021 

.548  .022 

.410  .022 

.404  .025 


WPN 

MEAN 

S.E. 

.181 

.016 

.335 

.019 

.320 

.018 

.758 

.020 

.561 

.028 

.577 

.020 
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SWAT  Score  Conversion 

The  purpose  of  developing  the  two  simulation  models 
was  to  modify  the  workload  estimate  in  the  Time  Load 
dimension  of  the  SWAT  methodology.  The  results  of  the 
computer  simulation,  as  described  in  the  previous  section, 
were  all  scaled  between  0  and  1.  To  make  these  compatible 
with  the  SWAT  methodology,  it  was  necessary  to  develop  a 
transformation  for  these  simulation-derived  estimates. 

Several  methods  were  considered  for  developing  this 
transformation,  but  the  method  used  was  to  develop  a  survey 
and  simply  ask  subjects  to  estimate  this  transformation. 

The  survey  form  used,  and  detailed  results,  are  found  in 
Appendix  G.  Final  results  are  summarized  in  Table  8. 

These  results  are  derived  from  82  subjects  who  completed 
the  survey.  Of  these,  46  were  pilots  and  36  were  non¬ 
pilots.  Four  of  the  subjects  were  also  recognized  experts 
in  workload  measurement,  while  the  other  78  were  not. 

Using  a  MANOVA  procedure  in  SAS  (35)  the  null 
hypothesis  of  no  overall  pilot  effect  was  tested  and  not 
rejected  (F  »  0.30,  P  >  F  ■»  0.93).  The  hypothesis  of  no 
overall  expert  effect  was  also  tested  and  not  rejected  (F  = 
1.34,  P  >  F  ■  0.25).  When  each  of  the  break  points  for  the 
simulation  output  parameters  was  tested  individually,  only 
the  experts'  estimate  of  the  break  point  between  LOW  and 
MEDIUM  levels  of  workload,  as  estimated  by  Pilot  Idle  Time, 
was  found  to  be  significant  (F  «  5.13,  P  >  F  »  0.026).  All 


Table  8  Simulation  Output,  SWAT  Score  Transformations 


SIMULATION 

OUTPUT 

PARAMETER 

PARAMETER  RANGES 

SWAT  CONVERSION 

PILOT  IDLE 

0%  <  PIR  <  12.3% 

HIGH  (3) 

RATE  (PIR) 

12.3%  <  PIR  <  44.1% 

MEDIUM  (2) 

44.1%  <  PIR  <  100% 

LOW  (1) 

TASK  INTERRUPT 

69.3%  <  TIR  <  100% 

HIGH  (3) 

RATE  (TIR) 

28.9%  <  TIR  <  69.3% 

MEDIUM  (2) 

0%  <  TIR  <  28.9% 

LOW  (1) 

SIMULTANEOUS 

65.5%  <  STR  <  100% 

HIGH  (3) 

TASK  RATE 

26.2%  <  STR  <  65.5% 

MEDIUM  (2) 

(STR) 

0%  <  STR  <  26.2% 

LOW  (1) 

other  individual  hypothesis  tests  proved  to  be  non¬ 
significant  at  the  5*  level. 

Because  the  experts  constituted  a  relatively  small 
group  and  because  the  hypothesis  of  no  overall  expert 
effect  was  not  rejected,  it  was  decided  to  use  all  survey 
responses  to  define  this  transformation.  Therefore,  the 
transformation  presented  in  Table  8  represents  the 
average  values  provided  by  all  82  survey  respondents. 
These  transformations  were  then  used  to  convert  the 
simulation  output,  found  in  Tables  6  and  7,  to  SWAT 
workload  predictions.  The  process  used  to  compare  these 
modified  predictions  to  the  RAN/ACE  SWAT  ratings  is 
described  in  the  next  section. 


86 

Predictor  Comparisons 

The  fundamental  question  to  be  addressed  in  this 
section  was  how  accurately  did  the  various  predictors  match 
the  SWAT  simulator  workload  ratings  of  the  RAH/ACE 
experiment.  When  Eggleston  and  Quinn  (7)  compared  their 
predicted  results  to  the  RAM/ACE  workload  measurements, 
they  used  simple  correlation.  This  was  possible  because 
they  could  compare  predictions  versus  ratings  for  specific 
pilots,  since  the  same  pilots  were  used  in  each  case. 
However,  in  this  research  a  totally  different  group  of 
pilots  was  used  to  predict  workload.  Therefore,  another 
approach  was  needed  which  allowed  comparison  of  predictions 
and  measurements  on  a  grouped,  rather  than  an  individual, 
basis. 

This  comparison  was  accomplished  by  viewing  the 
predictions  and  measurements  as  two  multivariate  sets  in  a 
two-way  experiment.  This  is  shown  in  Table  9.  This  table 
shows  that  the  workload  predictions  from  one  of  the  PRO¬ 
SWAT  data  sets  is  compared  against  SWAT  measurements  for 
all  three  cockpit  configurations  (ABL,  TF/TA,  and  TTA) . 

The  data  points  themselves  are  triplets,  i.e.,  workload 
values  for  the  Penetration  (P) ,  Target  Acquisition  (A) ,  and 
Weapon  Delivery  (W)  segments  for  each  combination  of 
cockpit  configuration  and  method.  When  viewed  this  way, 
the  multivariate  nature  of  the  data  is  more  obvious. 
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Table  9  Multivariate  Comparison  Process 


COCKPIT 

CONFIGURATION 

METHOD 

ABL 

TF/TA 

TTA 

PRO-SWAT  (1  or  2) 

P/A/W 

P/A/W 

P/A/W 

SWAT 

P/A/W 

P/A/W 

P/A/W 

A  table  of  information  such  as  this  could  be  formed 
for  each  predictor  and  each  PRO-SWAT  data  set.  The  SWAT 
workload  measurements  have  five  sets  of  pilot  ratings  in 
each  cell  of  the  matrix  above.  The  PRO-SWAT  estimates  made 
in  1982  also  have  five  sets  of  estimates  in  each  cell, 
while  those  completed  in  1986  have  eight  sets  in  each  cell. 

In  order  to  deal  with  this  data  using  multivariate 
techniques,  it  is  common  to  assume  that  the  data  has  a 
multivariate  normal  distribution.  Johnson  and  Wichem  (16) 
suggest  that  one  method  of  supporting  this  assumption  is  to 
test  the  marginal,  univariate  distributions  for  normality. 
This  can  be  done  by  developing  a  plot  of  the  ordered 
original  values  and  their  standard  normal  quantiles.  If 
these  plotted  values  form  a  straight  line,  the  assumption 
of  normality  is  supported.  However,  iiixS  requires 
subjective  judgment  as  to  the  *straightnessw  of  the 
resulting  line.  Pilliben  (8)  describes  a  probability  plot 
correlation  coefficient  test  for  normality.  This  test 
takes  the  subjective  element  out  of  the  process.  By 
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determining  a  correlation  coefficient  between  the  ordered 
data  and  their  standard  normal  quantiles,  and  comparing 
this  against  a  critical  value,  a  powerful  test  of  normality 
results.  This  test  was  done  for  each  univariate  data  set 
of  workload  predictions  and  measurements.  Of  the  153  sets 
checked,  only  six  sets  did  not  pass  this  univariate 
normality  check.  This  result  was  viewed  as  strong  support 
for  the  assumption  of  multivariate  normality. 

Multivariate  Comparison  of  Means 

Two  possible  methods  for  comparing  workload 
predictions  to  workload  measurements  were  identified.  The 
first  of  these  methods  is  a  comparison  of  multivariate 
means,  implemented  via  a  hypothesis  test: 

v  (1) 

where  JJ[ ^  is  a  3  x  3  matrix  of  mean  values  of  SWAT  workload 
measurements.  JU2  is  *  similar  ?  x  3  matrix  of  PRO-SWAT 
workload  predictions.  This  comparison  was  made  by 
comparing  one  predictor  (e.g.,  pilot  idle  time  from  the 
serial  processing  model)  at  a  time  against  the  SWAT 
workload  measurements.  Since  there  are  a  total  of  eight 
different  predictors  for  two  different  PRO-SWAT  sets,  there 
are  a  total  of  sixteen  hypothesis  tests. 
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There  are  three  basic  assumptions  which  must  be 
satisfied  for  this  analysis  to  be  valid.  The  first  is  that 
the  random  samples  from  different  populations  are 
independent.  This  independence  requirement  between  the 
SWAT  measurements  and  the  second  set  of  PRO-SWAT  estimates 
is  easily  satisfied  since  the  estimates  and  measurements 
were  accomplished  by  two  completely  distinct  groups  of 
pilots  approximately  four  years  apart.  The  original  PRO- 
SWAT  estimates  and  the  SWAT  ratings  were  made  by  the  same 
pilots  so  there  might  be  a  suspicion  that  these  data  sets 
are  not  independent.  However#  each  pilot  rated  workload 
(SWAT)  over  eight  different  simulator  flights  and  estimated 
workload  (PRO-SWAT)  over  the  same  number  of  missions. 

These  were  mixed  in  a  random  fashion  for  each  pilot#  so  an 
assumption  of  independence  seems  reasonable. 

The  second  assumption  is  that  the  covariance  matrix 
(£)  for  each  data  set  was  equal.  Johnson  and  Wichern  (16) 
suggest  that  any  discrepancy  of  the  order  of 

^  *  4 (fj  ^  is  probably  serious  and  should  cause 
concern.  There  were  no  values  with  this  large  a 
difference;  in  fact#  most  values  were  of  the  order  of 
^*1  ii  2  ^^2*  ii  or  *ess*  Bartlett  (1)  presents  a  formal 
test  to  statistically  verify  that  the  covariance  matricies 
are  equal. 

Bartlett's  test#  as  described  by  Green  (10)#  tests  the 
equivalence  of  covariance  matricies  by  computing  a  test 
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statistic  which  has  an  appropriate  chi  square  distribution. 
This  test  statistic  is  computed  by  the  formula: 


B 


=  (m-G)  In  |CW|  -  2  (®g  “  ln  |c< 


g=l 


(2) 


where :  m  =  ^  m 

g^i  9 

G  =  number  of  covariance  matricies 
m  =  number  of  observations  in  group  g 
Cw  =  pooled  covariance  matrix 
d.f.  =  1/2 [ (G-l) (p) (p+1) ] 
and  p  is  the  number  of  dependent  variables. 


(3) 

(4) 

(5) 

(6) 
(7) 


The  null  hypothesis,  that  the  covariance  matricies  are 
equal,  would  be  rejected  if  the  test  statistic,  B,  exceeds 
the  appropriate  chi  square  value  at  confidence  level  o(, 
Bartlett's  test  was  implemented  by  using  the  SAS  IML 
procedure  (Interactive  Matrix  Language)  (34).  All 
covariance  matricies  passed  Bartlett's  test,  so  it  is 
reasonable  to  assume  that  corresponding  covariance 
matricies  are  equal. 

The  final  assumption  was  that  each  population  is 
multivariate  normal.  This  assumption  was  satisfied,  as 
described  earlier  in  this  chapter. 

With  these  three  assumptions  satisfied,  it  is  possible 
to  conduct  this  hypothesis  test  with  confidence.  Table  10 
shows  the  results  of  this  hypothesis  test,  implemented  with 


Table  10  Multivariate  Comparison  of  Means 


PREDICTOR 

PRO-SWAT 

(1982) 

PRO-SWAT 

(1986) 

F 

F 

ezh 

PRO-SWAT 

0.33 

0.81 

1.26 

0.29 

SS-PIR 

0.32 

0.81 

2.03 

0.13 

SS-TI (A) 

1.27 

0.31 

7.40 

0.0006 

SS-TI(B) 

1.49 

0.24 

7.52 

0.0006 

SS-TI (C) 

1.06 

0.38 

3.29 

0.03 

SS-TI (D) 

1.28 

0.30 

2.32 

0,09 

PS-PIR 

0.44 

0.73 

2.34 

0.09 

PS -STR 

_ 

0.32 

0.81 

2.21 

0.11 

the  SAS  General  Linear  Models  Procedure  (35) . 

In  this  table  the  predictor,  PRO-SWAT,  refers  to  the 
purely  subjective  workload  estimates,  not  modified  by  any 
output  from  the  simulation.  All  predictors  beginning  with 
SS  were  derived  from  the  single  server  model  and  those 
beginning  with  PS  from  the  parallel  server  model.  PIR 
refers  to  pilot  idle  time  rate  and  TI  refers  to  one  of  the 
task  interrupt  rate  predictors.  Finally,  STR  refers  to  the 
simultaneous  task  rate. 

From  Table  10,  it  is  apparent  that  the  basic  PRO-SWAT 
data  from  1983  very  closely  matches  the  swat  measurements. 
The  1986  data  matches  the  SWAT  measurements,  but  not  nearly 
as  well.  This  is  not  surprising  and  the  reasons  for  this 
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will  be  discussed  in  the  next  chapter.  However,  the  data 
shows  that  when  the  1982  predictions  are  modified  with 
simulation  output,  the  F  statistic  generally  grows  but  the 
difference  between  means  never  becomes  significant.  Note 
that  the  pilot  idle  rate  (PIR)  predictor  for  the  single 
server  model  and  the  simultaneous  task  rate  (STR)  actually 
improve  the  basic  predictor  (PRO-SWAT)  slightly.  The 
unmodified  1986  data  does  not  match  the  SWAT  data  as  well 
as  the  1982  data,  but  is  not  significantly  different.  None 
of  the  predictors  improve  the  1986  data,  but  several  result 
in  differences  that  are  non-significant.  The  next  chapter 
will  present  some  conclusions  from  this  table. 

The  results  of  the  previous  table  were  derived  by 
comparing  the  output  for  a  given  predictor,  over  all  three 
cockpit  configurations,  to  the  SWAT  measurement  data.  A 
similar  analysis  was  also  made  by  considering  each  cockpit 
configuration  individually.  However,  this  method  of 
analysis  provided  no  discrimination  between  predictors  and 
did  not  reject  any  hypotheses  that  the  mean  predictor 
values  matched  the  SWAT  data.  This  apparent  inconsistency 
is  attributed  to  the  fewer  number  of  data  points  available 
when  the  hypothesis  was  tested  separately  for  each  cockpit 
configuration.  When  all  three  cockpit  configurations  were 
considered  simultaneously,  there  was  enough  information  not 
to  reject  the  hypothesis  for  some  and  reject  for  others. 


Profile  Analysis 
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The  second  multivariate  technique  used  to  evaluate  the 
predictors  was  profile  analysis,  as  described  by  Johnson 
and  wichern  (16) .  This  technique  tests  the  same 
hypothesis: 


V  Ai- A*  <•> 

This  test  is  accomplished  in  a  stagewise  fashion.  The 
first  stage  asks  the  question,  “Are  the  profiles  parallel?** 
An  auxiliary  hypothesis  could  be  formed: 


H0l>  C^-C^ 


(9) 


where  C  is  the  contrast  matrix: 


and  C  is  a  (p-1)  x  p  matrix.  In  this  case,  p  is  the  number 
of  treatments  (flight  segments)  for  each  population. 
Basically,  this  hypothesis  asks  if  a  plot  of  the  means  of 
each  population,  when  connected  by  straight  lines,  are 
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parallel.  This  still  allows  differences  in  mean  values, 
but  accepts  the  hypothesis  if  the  profiles  are  parallel. 
Hypothesis  HQ1  should  be  rejected  at  level  oC  if: 


/  / 


4-1, 


T  =(X!-X2)  C  C (1/n1+l/n2)CS^C  ]  ^C(XX-X2)  >  c~  (11) 


where  ^  and  X2  are  the  sample  mean  vectors,  Sp  is  the 
pooled  covariance  matrix,  and: 


2  (nx  +  n2  -  2)  (p  -  1) 
c  =  —  - - - - 

nl  +  n2  "P 


Fp-l,nl+n2-p^°^  ^12) 


The  second  stage  asks  the  question,  "Assuming  the 
profiles  are  parallel,  are  they  coincident?"  Another 
auxiliary  hypothesis  could  be  formed: 

H02:  IJh-k/lz  <13> 


where  1  =[11--  -  11] 


(14) 


Stated  another  way,  given  that  the  profiles  are  parallel, 
do  they  lie  on  top  of  one  another?  These  two  hypotheses 
are  germane  to  this  research  since  they  test  how  closely 
the  means  of  the  two  populations  agree.  Hypothesis  H, 
should  be  rejected  at  level  Xif: 


*02 


r*  =  i/(x1~x2H(1/n1+i/n2)i/spi]"1i,(X1-x2)  > 


l,nl+n2-2 


(15) 
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The  third  stage  of  the  profile  analysis  asks  the 
question,  "Assuming  the  profiles  are  coincident,  are  the 
profiles  level?"  Stated  as  a  third  auxiliary  hypothesis: 

H03:  Cl^  +  Mi)  -  0  (16) 

Hypothesis  HQ3  should  be  rejected  at  level  if: 

(n1+n2!X/C/rC3pC,]-1C)C  >  Pp_1(  nl+BJ_p<*)  <»> 

where  X  ~  n^^  +  n?)  +  n2  /(n^  +  n2)  X2  (18) 

While  tb‘M  c  drd  test  may  be  important  in  some  domains,  it 
was  not  considered  important  in  this  research.  There  is  no 
reason  to  suspect,  for  example,  that  workload  in  all  three 
flight  phases,  or  all  three  cockpit  configurations,  should 
be  the  same.  In  fact,  there  are  many  reasons  to  expect 
just  the  opposite  is  true. 

The  first  attempt  at  conducting  a  profile  analysis 
considered  each  predictor  over  all  cockpit  configurations. 
However,  this  proved  impossible.  To  compute  the 
appropriate  test  statistic  required  a  9  x  9  covariance 
matrix  (three  flight  phases  for  three  cockpit 
configurations) .  The  data  available  for  this  was  limited 
to  only  five  data  points  for  both  the  SWAT  and  1981  PRO¬ 
SWAT  data,  and  eight  data  points  for  the  1986  data.  This 


A  -V  A  iV  I3i  L%  '_y  IS  VS  _S  IV  A  i>VS 


led  to  covariance  matricies  of  less  than  full  rank. 
Therefore,  it  was  impossible  to  test  the  hypothesis  on  this 
basis.  The  second  attempt  at  a  profile  analysis  was 
conducted  separately  for  each  cockpit  configuration.  This 
resulted  in  a  3  x  3  covariance  matrix.  Since  there  were  a 
minimum  of  five  data  points  in  each  cell,  the  resulting 
covariance  matrix  was  of  full  rank  and  the  analysis  could 
be  conducted.  Sample  profiles  are  shown  in  Figures  9  and 

10  for  the  PRO-SWAT  1986  data  and  the  1982  data, 
respectively.  The  results  of  this  profile  analysis  are 
shown  in  Table  11  for  the  1982  PRO-SWAT  data  and  Table  12 
for  the  1986  PRO-SWAT  data. 

The  predictors  in  both  of  these  tables  are  defined  in 
the  same  way  as  they  were  for  Table  10.  The  entries  in  the 
tables  are  the  values  of  the  test  statistics  computed  by 
Equations  11  (HQ1),  15  (HQ2),  and  17  (H03) ,  Also  shown  are 
the  critical  values,  which  can  be  compared  to  the  test 
statistics.  Where  the  test  statistic  exceeds  the  critical 
value,  the  corresponding  hypothesis  should  be  rejected  for 
that  predictor.  As  shown  in  Table  11,  neither  the 
hypothesis  for  parallel  profiles  nor  the  hypothesis  for 
coincident  profiles  is  rejected  for  the  basic  data  (PRO- 
SWAT)  or  any  of  the  predictors.  The  last  column  of  Table 

11  shows  Bartlett's  test  statistic  and  its  corresponding 
critical  value.  As  discussed  earlier,  Bartlett's  test  was 
used  to  determine  if  the  covariance  matricies  of  the 
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Table  11  Profile  Analysis  Results,  1982  Data 


CONFIG¬ 

URATION 

PREDIC¬ 

TOR 

PARALLEL 

H01 

COINCIDENT 

H02 

EQUAL 

H03 

BARTLETT 

STATISTIC 

ABL 

PRO-SWAT 

2.35 

0.20 

8.06 

4.07 

SS-PIR 

4.63 

0.58 

10.46 

11.33 

SS-TI(A) 

1.96 

0.06 

6.75 

12.54 

SS-TI(B) 

2.79 

0.14 

6.41 

11.71 

SS-TI(C) 

1.38 

1.27 

10.21 

10.91 

SS-TI (D) 

4.53 

0.54 

10.30 

10.62 

PS-PIR 

1.77 

0.02 

8.67 

10.67 

PS-STR 

4.63 

0.58 

10.46 

11.33 

TF/TA 

PRO-SWAT 

3.84 

0.42 

7.10 

11.71 

SS-PIR 

1.99 

0.79 

8.71 

3.75 

SS-TI(A) 

2.32 

1.97 

7.51 

4.13 

SS-TI(B) 

2.32 

1.97 

7.51 

4.13 

SS-TI(C) 

0.18 

0.06 

15,38 

3.82 

SS-TI(D) 

0.06 

0.24 

16.44 

3.80 

PS-PIR 

0.65 

1.29 

11.29 

4.05 

PS-STR 

1.99 

0.79 

8.71 

3.75 

TTA 

PRO-SWAT 

0.11 

0.01 

9.42 

1.85 

SS-PIR 

0.08 

0.24 

8.66 

1.26 

SS-TI (A) 

0.26 

2.97 

6.98 

1.22 

SS-TI(B) 

0.26 

2.98 

6.98 

1.22 

SS-TI (C) 

1.35 

1.07 

11.19 

1.03 

SS-TI (D) 

1.35 

1.07 

11.19 

1.03 

PS-PIR 

0.08 

0.24 

8.66 

1.26 

PS-STR 

0.08 

0.24 

8,66 

1.26 

CRITICAL  VALUE 

10.83 

5.32 

4.74 

12.59 
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predictor  data  and  the  corresponding  SWAT  data  were  equal. 
In  all  cases,  the  equality  of  covariance  matricies  was 
established.  This  equality  satisfies  one  of  the  three 
assumptions  mentioned  earlier  and  is  a  critical  assumption 
when  sample  sizes  are  small,  as  they  are  in  this  research. 

Some  additional  comments  are  in  order  regarding  the 
data  in  Table  11.  Note  that  the  original  PRO-SWAT 
predictions,  over  all  three  cockpit  configurations,  are 
excellent.  In  fact,  there  are  relatively  few  predictors 
which  improve  the  predictions  in  terms  of  being  both 
parallel  and  coincident.  There  appears  to  be  no 
discernable  pattern  as  to  which  predictor  consistently 
improved  or  worsened  the  estimates  over  the  three  cockpit 
configurations.  Perhaps  even  more  important,  none  of  the 
predictors  changed  the  original  PRO-SWAT  estimates  so  much 
that  they  were  rejected  by  the  parallel  or  coincident 
hypothesis  tests.  Also  note  that  all  predictors  were 
rejected  by  HQ3,  testing  whether  the  profiles  were  level. 
In  essence,  this  suggests  that  workload  levels  over  the 
three  flight  segments  were  different  and  this  difference 
was  recognized  by  the  statistical  test. 

In  general,  the  1986  PRO-SWAT  data  does  not  predict 
the  SWAT  measurements  as  accurately  as  the  1982  PRO-SWAT 
data.  This  is  shown  in  Table  12,  which  depicts  results  of 
the  profile  analysis  on  this  data.  The  hypothesis  of 
parallel  profiles  (HQ1)  is  rejected  for  two  of  the  task 
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Table  12  Profile  Analysis  Results,  1986  Data 
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CONFIG¬ 

URATION 

PREDIC¬ 

TOR 

PARALLEL 

H01 

COINCIDENT 

H02 

EQUAL 

»03 

BARTLETT 

STATISTIC 

ABL 

PRO-SWAT 

2.82 

0.02 

2.07 

4.34 

SS-PIR 

2.11 

0.35 

8.17 

8.51 

SS-TI(A) 

3.84 

3.02 

3.31 

6.38 

SS-TI(B) 

3.84 

3.02 

3.31 

6.38 

SS-TI(C) 

0.68 

1.98 

13.17 

9.32 

SS-TI(D) 

2.11 

0.35 

8.17 

8.51 

PS-PIR 

1.23 

0.55 

7.08 

5.18 

PS-STR 

2.17 

0.20 

7.37 

9.14 

TF/TA 

PRO-SWAT 

8.07 

0.04 

0.94 

4.10 

SS-PIR 

5.50 

0.13 

1.84 

6.61 

SS-TI(A) 

9.40 

1.84 

1.23 

6.45 

SS-TI(B) 

9.40 

1.84 

1.23 

6.45 

SS-TI(C) 

4.22 

0.74 

6.89 

4.38 

SS-TI(D) 

1.31 

0.19 

9.02 

6.55 

PS-PIR 

5.83 

0.76 

4.08 

6.48 

PS-STR 

5.50 

0.13 

1.84 

6.61 

TTA 

PRO -SWAT 

3.66 

0.14 

2.45 

7.49 

SS-PIR 

1.62 

0.04 

6.89 

3.65 

SS-TI(A) 

3.80 

2.46 

2.41 

5.36 

SS-TI(B) 

3.87 

3.15 

2.29 

5.51 

SS-TI(C) 

1.43 

4.10 

8.48 

7.62 

SS-TI(D) 

1.43 

4.10 

8.48 

7.62 

PS-PIR 

1.62 

0.04 

6.89 

3.65 

PS-STR 

1,62 

0.04 

6.89 

3.65 

CRITICAL  VALUE 

9.02 

4.84 

4.10 

12.59 
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interrupt  predictors  (SS-TI(A)  and  SS-TI(B))  for  the  TF/TA 
configuration.  This  result  is  consistent  with  the  results 
shown  in  Table  10 ,  Multivariate  Comparison  of  Means,  which 
also  shows  these  predictors  to  produce  results  different 
than  the  SWAT  measurements  at  the  .0006  level  of 
significance. 

For  the  remaining  predictors,  the  hypotheses  of 
parallel  and  coincident  profiles  could  not  be  rejected  for 
the  1986  PRO-SWAT  data  at  0.05.  This  result  is 
apparently  contradictory  to  the  results  obtained  from  the 
multivariate  comparison  of  means.  Table  10,  That  table 
also  showed  that  the  PRO-SWAT,  SS-TI(D),  and  PS-STR 
predictors  were  significantly  different  at  anO^  *  0.05. 
once  again,  the  inconsistency  between  these  tables  should 
be  attributed  to  differences  in  the  amount  of  data 
available  for  each  test.  The  multivariate  comparison  of 
means  had  more  data  available  and  was  more  discriminating. 

The  profile  analysis  of  the  1986  PRO-SWAT  data  also 
showed  that  the  hypothesis  of  equal  workload  levels  (HQ3) 
was  not  rejected  in  9  of  22  cases  considered.  This  result 
does  not  include  the  SS-TI(A)  and  SS-TI(B)  predictors  in 
the  TF/TA  configuration  since  they  had  already  been 
rejected  by  the  parallel  profile  hypothesis.  Recall  that 
profile  analysis  is  a  stagewise  succession  of  tests  that 
determine  if  multivariate  means  are  equal.  Failure  of  the 
first  stage  automatically  causes  failure  of  subsequent 


stages  because  each  test  is  conditional  on  passing  the 
previous  test.  The  fact  that  9  of  22  cases  were  identified 
as  having  equal  levels  of  workload  is  alarming  since  it  is 
known  that  the  workload  at  each  of  these  levels  was,  in 
fact,  different.  This  indicates  that,  while  the  hypotheses 
of  being  parallel  and  coincident  could  not  be  rejected,  the 
1986  PRO-SWAT  data  was,  in  some  cases,  relatively  flat  over 
the  three  flight  phases.  This  can  be  seen  graphically  in 
Figure  3. 

Sensitivity  of  Results 

The  final  segment  of  this  chapter  looks  at  the 
sensitivity  of  results  to  some  of  the  assumptions  made  in 
conducting  this  research.  Three  basic  assumptions  are 
discussed.  The  first  of  these  relates  to  the  definition  of 
a  task  interruption,  as  described  by  the  three  task 
interrupt  predictors.  The  second  assumption  tested  is  that 
of  ignoring  a  significant  difference  of  workload  experts' 
estimate  of  the  transformation  of  simulation  output  results 
to  SWAT  scores.  The  third  assumption  tested  is  the 
decision  to  require  non-preempting  tasks  to  queue  for 
service  if  the  service  time  degradation  factor  exceeds  an 
arbitrary  value  of  1.0.  This  assumption  applies  only  tc 
the  parallel  processing  model. 


Task  Interruption  Definition 


This  research  considered  the  concept  of  task 
interruption  in  the  serial  processing  model  as  a  surrogate 
for  the  Time  Load  dimension  of  the  SWAT  methodology.  Four 
different  definitions  of  task  interruption  were  provided 
and  measured.  Table  4  defined  each  of  these  predictors. 
Predictors  A  and  B  were  similar  in  that  a  task  interruption 
occurred  when  a  task  was  preempted  by  another.  Predictors 
C  and  D  recognized  a  task  interruption  when  a  new  task 
arrived  and  the  pilot  was  already  occupied  with  another, 
regardless  of  whether  a  preemption  occurred.  Predictors  A 
and  C  counted  subsequent  arrivals  of  preempted  tasks  as 
additional  arrivals,  while  predictors  B  and  D  did  not. 

From  Table  6  it  is  evident  that  there  is  little 
difference  between  simulation  output  for  predictors  A  and 
B.  The  same  results  can  be  drawn  from  Table  6  about  the 
difference  between  C  and  D.  This  suggests  that  counting  or 
not  counting  the  subsequent  arrival  of  preempted  tasks  had 
little  effect  on  the  predictors.  However,  there  does 
appear  to  be  significant  differences  between  predictors  A 
and  C  as  shown  by  Table  6.  A  similar  observation  can  be 
made  about  predictors  B  and  D.  In  both  cases,  counting 
preemptions  as  task  interruptions  produced  significantly 
lower  estimates  of  the  task  interruption  rate  than  did 
counting  arrivals  of  new  tasks  while  another  task  was 


already  being  worked.  This  is  logical  and  intuitively 
appealing. 

The  obvious  question  is  which  of  these  two  options 
provides  the  more  accurate  prediction.  Using  the 
multivariate  comparison  of  means  technique,  as  presented  in 
Table  10,  significant  differences  are  shown  for  the  1986 
PRO-SWAT  data.  Predictors  A  and  B  show  the  most 
significant  differences  from  the  SWAT  measurements  and 
strongly  suggest  that  using  the  notion  of  task  preemption 
as  a  definition  of  a  task  interruption  is  inappropriate. 
Predictor  D  shows  non-significant  differences  (at  the  5% 
level)  from  the  SWAT  measurement  data.  Predictor  C  shows 
significant  differences  at  the  5%  level.  This  result 
suggests  that  counting  task  arrivals  while  another  task  is 
being  worked  is  a  better  definition  of  task  interruption. 
However,  this  result  is  mixed  and  perhaps  the  better 
predictor  is  achieved  when  Predictor  D  is  used.  That  is, 
count  task  arrivals  when  another  task  is  being  worked  as  an 
interrupt  but  do  not  count  subsequent  arrivals  of  preempted 
tasks  as  new  arrivals. 

The  1982  PRO-SWAT  data  provides  very  little  evidence 
about  the  definition  of  task  interruptions  as  shown  in 
Table  10.  Since  none  of  these  predictors  are  rejected  and 
all  statistics  have  about  the  same  value,  little  can  be 
said  about  the  predictors. 
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The  profile  analysis  of  the  1986  PRO-SWAT  data,  as 
shown  in  Table  12,  partially  confirms  the  results  of  the 
previous  two  paragraphs.  For  the  TF/TA  configuration, 
Predictors  A  and  B  are  shown  to  produce  profiles  non¬ 
parallel  to  the  SWAT  measurement  data.  The  results  for 
Predictors  C  and  D  were  mixed  in  this  table,  although  both 
were  not  rejected  by  either  the  parallel  or  coincident 
hypothesis  test. 

The  1982  PRO-SWAT  data,  as  shown  in  Table  11,  did  not 
reject  any  of  the  predictors.  In  fact,  it  provided  only 
mixed  results  regarding  the  relative  accuracy  of  the  four 
predictor  definitions. 

Workload  Expert's  Transformation 

As  described  earlier  in  this  chapter,  a  transformation 
was  developed  to  convert  simulation  output  to  SWAT  1-2-3 
ratings.  As  was  noted,  a  significant  difference  in  the 
expert's  estimate  of  the  dividing  line  between  LOW  and 
MEDIUM  levels  of  workload,  as  estimated  by  Pilot  Idle  Time, 
was  ignored.  However,  had  the  experts'  opinion  been  used, 
only  two  transformations  would  have  changed.  These 
transformations  are  for  the  single  server  model.  Pilot  Idle 
Time  predictor.  Penetration  flight  phase,  and  for  the  TF/TA 
and  TTA  cockpit  configurations.  In  both  cases,  a  LOW  Time 
Load  workload  estimate  was  changed  to  a  MEDIUM  estimate. 

All  other  predictor  transformations  remained  unchanged. 
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The  effect  of  these  changes  was  measured  by 
reaccomplishing  both  the  multivariate  comparison  of  means 
and  the  profile  analysis.  The  results  of  the  former  method 
are  shown  in  Table  13.  Although  the  numbers  change  for  the 
1982  PRO-SWAT  data,  the  conclusion  is  the  same.  However, 
for  the  1986  PRO-SWAT  estimates,  using  the  experts’ 
transformation  leads  to  a  significant  difference  for  this 
predictor. 


Table  13  Transformation  Sensitivity,  Multivariate 
Comparison  of  Means  (Predictor  »  SS-PIR) 


Transformation 

PRO-SWAT 

(1^82) 

PRO-SWAT 

(1986) 

F 

P  >  F 

F 

P  >  F 

Original 

0.32 

0.81 

2,03 

0.13 

Expert 

0.64 

0.60 

4.09 

0.01 

Table  14  shows  the  result  of  using  the  expert’s 
transformation  for  the  profile  analysis.  With  the  expert’s 
transformation,  the  parallel  profile  hypothesis  is  rejected 
at  the  0.05  level  for  the  1986  TF/TA  data.  Once  again, 
this  is  for  the  Pilot  Idle  Time  predictor.  This  is 
consistent  with  the  result  of  the  multivariate  comparison 
of  means  findings,  just  discussed,  and  shown  in  Table  13. 

In  general,  this  method  is  not  sensitive  to  not  using  the 
experts*  transformation.  However,  it  is  apparent  that  use 
of  this  transformation  did  cause  the  rejection  of  a 


Table  14  Transformation  Sensitivity,  Profile  Analysis 
(Predictor  =  SS-P.TM 


Transfor- 

Method/ 

Parallel 

Coincident 

Equal 

Bartlett 

mation 

Config 

H01 

H02 

»03 

Statistic 

Original 

82/TF/TA 

1.99 

0.79 

8.71 

3.75 

82/TTA 

0.08 

0.24 

8.66 

1.26 

86/TF/TA 

5.50 

0.13 

1.84 

6.61 

86/TTA 

1.62 

0.04 

6.89 

3.65 

Expert 

82/TF/TA 

2.15 

0.43 

7.67 

3.73 

82/TTA 

0.20 

0.001 

7.02 

1.06 

86/TF/TA 

9.40 

0.01 

1.23 

4.52 

86/TTA 

3.88 

0.68 

2.28 

7.02 

predictor  which  had  not.  been  rejected  by  the  original 
transformation , 

Service  Time  Degradation  Factor 

The  parallel  processing  model  allows  simultaneous 
processing  of  tasks  up  to  a  limit.  However,  as  each  new 
task  is  added  to  any  tasks  already  being  worked,  the 
remaining  service  times  of  e'il  involved  tasks  were 
increased.  The  amount  of  this  increase  was  determined  by 
computing  a  degradation  factor  based  on  North’s  conflict 
matrix  (23)  .  This  conflict  matrix  was  based  on  the  number 
of  resource  demands  c  .'ssd  by  the  tasks  involved.  When  the 
degradation  factor  exceeded  an  arbitrary  limit  of  1.0,  the 
new  task  was  placed  in  a  queue  to  wait  for  service  if  it 


could  not  preempt  another  task.  The  1.0  limit  represented 
a  doubling  of  service  times,  so  it  was  arbitrarily  selected 


as  the  upper  limit  of  the  degradation  factor. 

To  examine  the  sensitivity  of  this  arbitrary  decision, 
the  degradation  limit  was  varied  and  the  parallel 
processing  model  was  rerun  for  all  three  flight 
configurations.  The  results  are  shown  in  Table  15. 

Several  conclusions  are  obvious  from  this  table.  First, 
pilot  idle  time  ratio  is  relatively  insensitive  to  changes 
in  the  degradation  limit  over  all  three  cockpit 
configurations  and  three  flight  phases.  This  is  not 
surprising.  Changing  the  degradation  limit  does  not  change 
the  number  of  tasks  that  arrive,  only  the  number  that  can 
be  processed  simultaneously.  Decreasing  the  degradation 
limit  places  more  tasks  in  the  queue  for  later  processing, 
which  would  tend  to  decrease  pilot  idle  time.  However, 
this  is  compensated  for  by  the  lower  service  times  for 
those  tasks  actually  being  worked.  Hence,  pilot  idle  time 
ratio  is  relatively  insensitive  to  changes  in  the 
degradation  limit. 

Table  15  shows  that  the  simultaneous  task  rate  is  more 
sensitive  to  changes  in  the  degradation  limit.  This  is  not 
surprising  because,  as  the  degradation  limit  is  decreased, 
fewer  tasks  are  worked  simultaneously.  Therefore,  the 
simultaneous  task  rate  will  decrease  as  the  degradation 
limit  decreases.  Except  for  minor  variations  due  to  random 
effects,  this  trend  was  shown  consistently  over  all  cockpit 
configurations  and  flight  phases. 


TABLE  15  Service  Time  Degradation  Limit  Sensitivity 


CONFIG¬ 

DEG 

IDLE 

TIME  RATIO  ] 

SIMULTANEOUS  TASK  RATE 

URATION 

LIMIT 

PEN 

ACQ 

WPN 

PEN 

ACQ 

WPN 

ABL 

0.6 

0.55 

0.25 

0.11 

0.19 

0.23 

0.37 

0.8 

0.57 

0.32 

0.15 

0.26 

0.42 

0.59 

0.57 

0.32 

0.18 

0.35 

0.55 

0.76 

flSfl 

0.55 

0.30 

0.14 

0.38 

0.58 

0.84 

Ba 

0.55 

0.30 

0.14 

0.38 

0.58 

0.84 

TF/TA 

0.6 

0.68 

0.40 

0.32 

0.18 

0.16 

0.39 

0.8 

0.68 

0.42 

0.33 

0.21 

0.27 

0.49 

0.68 

0.45 

0.33 

C.26 

0.41 

0.56 

iwi 

0.68 

0.45 

0.31 

0.26 

0.41 

0.58 

mm 

0.68 

0.44 

0.31 

0.25 

0.41 

0.58 

TTA 

0.6 

0.66 

0.39 

0.30 

0.19 

0.16 

0,36 

0.8 

0.66 

0.41 

0.32 

0.20 

0.30 

0.48 

■ 

0.66 

0.44 

0.32 

0.24 

0.40 

0.58 

■ss 

0.66 

0.45 

0.28 

0.25 

0.42 

0.60 

■39 

0.66 

0.45 

0.28 

0.25 

0.42 

0.60 

The  service  time  degradation  limits  shown  in  Table  15 
were  also  translated  to  SWAT  workload  scores  for  both  the 
1982  and  1986  prediction  sets.  These  predictions  were  then 
compared  to  the  SWAT  workload  measurements.  Using  the 
multivariate  comparison  of  means,  the  results  of  this 
comparison  are  shown  in  Table  16.  Note  that  degradation 
limits  above  1.0  are  not  reported  since  both  predictors 
produced  results  equal  to  those  of  a  degradation  factor  of 
1.0  for  all  values  greater  than  1.0.  This  is  not 
surprising  since  North's  conflict  matrix  (22),  as  shown  in 
Table  2,  produces  very  few  degradation  factors  greater  than 
1,0  and  none  greater  than  1.1. 

The  results  shown  in  Table  16  indicate  that,  for  the 
simultaneous  task  rate  predictor,  a  degradation  factor  of 


1.0  produced  the  most  accurate  predictions.  For  the  pilot 
idle  time  predictor  and  1982  PRO-SWAT  data,  there  was  no 
significant  change  in  results  over  the  range  of  degradation 
factors.  However,  for  the  1986  PRO-SWAT  data,  workload 
predictions  more  closely  matched  measurements  as  the 
degradation  limit  decreased.  This  was  exactly  opposite  to 
the  trend  for  the  Simultaneous  Task  Rate  predictor 
discussed  previously. 


Table  16  Degradation  Limit  Sensitivity,  Multivariate 
Comparison  of  Means 


PREDICTOR 

DEG. 

LIMIT 

1982 

1986 

F 

P  >  F 

F 

P  >  F 

PS-PIR 

0.6 

0.49 

0.70 

1.50 

0.23 

0.8 

0.49 

0.70 

2.40 

0.09 

1.0 

0.44 

0.73 

2.34 

0.09 

PS-STR 

0.6 

1.29 

0.30 

3.77 

0.02 

0.8 

0.44 

0.72 

2.47 

0.08 

1.0 

0.32 

0.81 

2.21 

0.11 

When  degradation  limit  sensitivity  is  assessed  using 
profile  analysis,  the  results  are  inconclusive.  Tables  17 
and  18  show  these  results  and  there  appears  to  be  no 
consistent  trend  over  both  1982  and  1986  PRO-SWAT  data 
sets.  Once  again,  the  test  statistics  displayed  under  the 
PARALLEL  and  COINCIDENT  columns  are  the  ones  most  critical 
to  determining  how  well  SWAT  predictions  match 
measurements . 


m 
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Table  17  Degradation  Limit  Sensitivity,  Profile  Analysis 
1982  Data 


i'iTI' 

r 


$ 


CONFIG¬ 

PREDIC¬ 

DEG 

PARALLEL 

COIN. 

EQUAL 

BARTLETT 

URATION 

TOR 

LIMIT 

H01 

H02 

H03 

STATISTIC 

ABL 

PS-PIR 

0.6 

1.77 

0.02 

8.67 

10.67 

0.8 

1.77 

0.02 

8.67 

10.67 

1.0 

1.77 

0.02 

8.67 

10.67 

PS-STR 

0.6 

3.88 

0.01 

8.75 

11.05 

0.8 

1.77 

0.02 

8.67 

10.67 

1.0 

4.63 

0.58 

10.46 

11.33 

TF/TA 

PS-PIR 

0.6 

0.65 

1.29 

11.99 

4.05 

0.8 

0.65 

1.29 

11.99 

4.05 

1.0 

0.65 

1.29 

11.99 

4.05 

PS-STR 

0.6 

0.65 

1.29 

11.99 

4.05 

0,8 

1.99 

0.79 

8.71 

3.75 

1.0 

1.99 

0.79 

8.71 

3.75 

TTA 

PS-PIR 

0.6 

0.14 

1.26 

7.01 

1.21 

0.8 

0.14 

1.26 

7.01 

1.21 

1.0 

0.08 

0.24 

8.66 

1.26 

PS-STR 

0.6 

0.83 

1.15 

9.19 

1.27 

0.8 

0.83 

1.15 

9.19 

1.27 

1.0 

0.08 

0.24 

8.66 

1.26 

CRITICAL  VALUE 

10.83 

5.32 

4.74 

12.59 

Table  18  Degradation  Limit  Sensitivity,  Profile  Analysis 
1986  Data 


CONFIG- 

PREDIC- 

DEG 

PARALLEL 

COIN. 

EQUAL  BARTLETT 

URATION 

TOR 

LIMIT 

H01 

H02 

HQ3  STATISTIC 

ABL 

PS-PIR 

0.6 

0.54 

0.003 

11.03 

5.50 

00 

• 

o 

1.32 

0.60 

7.06 

5.34 

1.23 

0.55 

7.08 

5.18 

PS-STR 

2.33 

1.66 

4.98 

5.86 

0.8 

1.32 

0.59 

7.07 

5.30 

1.0 

2.17 

0.20 

7.37 

9.14 

TF/TA 

PS-PIR 

0.6 

5.83 

0.76 

4.08 

6.48 

0.8 

5.83 

0.76 

4.08 

6.48 

1.0 

5.83 

0.76 

4.08 

6.48 

PS-STR 

0.6 

5.83 

0.76 

4.08 

6.48 

0.8 

5.50 

0.13 

1.84 

6.61 

1.0 

5.50 

0.13 

1.84 

6.61 

TTA 

PS-PIR 

0.6 

1.62 

0.04 

6.89 

3.65 

0.8 

1.62 

0.04 

6.89 

3.65 

1,0 

1.62 

0.04 

6.89 

3.65 

PS-STR 

0.6 

2.00 

0.50 

4.82 

5.37 

0.8 

2.00 

0.50 

4.82 

5.37 

1.0 

1.62 

0.04 

6.89 

3.65 

CRITICAL  VALUE 


9.02 


4.84 


4.10  12.59 
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Summary 

This  chapter  presented  the  results  of  this  research, 
comparing  two  predictor  data  sets  to  SWAT  measurements  and 
modifying  these  predictor  sets  to  develop  additional 
predictors.  These  modifications  were  based  on  the  output 
from  computer  simulation  models.  The  process  which 
transformed  the  simulation  output  to  SWAT  scores  was  also 
described. 

Using  two  means  of  evaluation,  a  multivariate 
comparison  of  means  and  profile  analysis,  all  predictors 
were  compared  to  SWAT  measurements.  Some  predictors  were 
found  to  improve  the  unmodified  predictors  and  others  did 
not.  In  general,  the  predictor  values  matched  SWAT 
measurements  reasonably  well. 

Based  on  the  multivariate  comparison  of  means,  the 
most  accurate  predictors  are  Pilot  Idle  Time,  using  the 
single  server  model,  and  Simultaneous  Task  Rate,  using  the 
parallel  processing  model.  Almost  as  accurate  is  Pilot 
Idle  Time,  using  the  parallel  processing  model.  Although 
the  results  are  not  as  obvious,  profile  analysis  generally 
identifies  the  same  three  predictors  as  producing  the  most 
accurate  predictions  over  all  cockpit  configurations  and 
both  predictor  data  sets.  Since  both  the  single  server 
model  and  the  parallel  processing  model  produced  a 
predictor  with  approximately  equal  accuracy,  it  is  not 
possible  to  state  that  one  model  is  more  accurate  than 


another.  It  does  appear  that  the  parallel  processing  model 
may  be  more  consistent,  since  both  parallel  processing 
predictors  are  relatively  accurate.  The  single  server 
model  produced  mixed  results,  a  relatively  accurate  Pilot 
Idle  Rate  predictor  and  an  inaccurate  Task  Interruption 
Rate  predictor. 

The  sensitivity  of  several  assumptions  was  also 
investigated.  First,  several  different  definitions  for 
task  interruptions  were  tested.  It  was  found  that  defining 
a  task  interruption  as  the  arrival  of  a  new  task  which 
occurred  while  the  pilot  was  busy  produced  the  most 
accurate  prediction  for  this  predictor.  The  second 
assumption  tested  was  the  validity  of  ignoring  a  different 
estimate  of  idle  time  categories  posed  by  workload  experts. 
It  was  shown  that  using  the  experts'  estimate  actually 
worsened  the  prediction. 

The  third  assumption  tested  was  the  sensitivity  to  an 
arbitrary  service  time  degradation  limit  of  1.0  in  the 
parallel  processing  model.  It  was  shown  that  the  Pilot 
Idle  Time  predictor  was  insensitive  to  this,  but  the 
Simultaneous  Task  Rate  was  somewhat  sensitive  to  this. 

The  next  chapter  reviews  these  results  and  draws 
conclusions  about  this  research.  Following  these,  some 
recommendations  are  made  for  additional  work  which  will 
improve  the  methods  described. 


Chapter  VI 

CONCLUSIONS  AND  RECOMMENDATIONS 

Introduction 

The  first  chapter  stated  that  the  intent  of  this 
research  was  to  develop  and  validate  a  model  which  predicts 
pilot  workload.  Chapters  II  through  IV  discussed  the 
development  of  this  model  and  Chapter  V  presented  the 
model's  predictions.  This  chapter  will  summarize  these 
results  and  draw  conclusions.  These  conclusions  will  be 
about  the  modeling  process  as  a  whole,  as  well  as  about 
specific  predictors  and  assumptions  of  the  model.  Finally, 
some  recommendations  for  further  study  will  be  presented. 

Conclusions 

In  general,  it  appears  that  the  model  developed  in 
this  research  predicts  workload  reasonably  well.  However, 
some  of  the  assumptions  and  techniques  used  produced  more 
accurate  results  than  other  options.  Furthermore,  all 
predictions  were  measured  against  a  single  set  of 
measurements.  Before  full  acceptance  of  this  method, 
predictions  should  also  be  compared  against  other  sets  of 
workload  measurement  data. 


Before  presenting  specific  conclusions,  a  brief 
discussion  of  the  basic  PRO-SWAT  workload  estimates  is  in 
order.  Two  sets  of  workload  predictions  were  used  in  this 
research:  one  set  developed  in  1982  and  one  set  in  1986. 
Although  every  attempt  was  made  to  derive  accurate 
predictions,  it  appears  that  those  developed  in  1986  were 
not  as  accurate  as  the  1982  set.  There  are  several  reasons 
for  this.  First,  the  administrators  of  the  1982  experiment 
were  experienced  in  the  SWAT  measurement  technique.  This 
was  not  true  for  the  1986  predictions.  The  1986 
predictions  were  administered  by  the  author  on  his  first 
and  only  experience  with  developing  SWAT  predictions.  In 
both  cases,  however,  the  same  materials  were  used  by  the 
pilots  and  the  same  instructions  given  to  the  pilots. 

The  second  major  reason  for  differences  in  the  two 
sets  of  predictions  concerns  the  pilots  used  in  each 
exercise.  The  pilots  making  predictions  in  1982  were 
qualified  F-15  pilots  who  were  asked  to  estimate  workload 
in  an  advanced  version  of  the  F-15.  Furthermore,  these 
pilots  were  active  duty  Air  Force  pilots  who  were  assigned 
to  participate  in  this  exercise  for  several  weeks. 
Therefore,  the  pilots  were  able  to  dedicate  time  and 
attention  to  accomplishing  this.  The  predictions  developed 
in  1986  were  accomplished  by  Air  National  Guard  pilots, 
qualified  in  the  A-7  aircraft.  They  did  not  fly,  nor  had 


they  ever  flown  the  F-15.  These  pilots  volunteered  to 
participate  in  the  study,  but  did  so  in  their  spare  time. 

In  other  words,  they  were  not  freed  of  other  duties  to  work 
on  this  study.  The  A-7  pilots  made  all  workload  estimates 
on  their  own,  without  any  means  of  monitoring  their  effort 
or  any  chance  of  asking  questions. 

The  accuracy  of  the  results  could  have  been  predicted. 
The  1982  PRO-SWAT  estimates  were  more  accurate  than  the 
1986  estimates.  However,  from  the  perspective  of  this 
research,  that  may  have  been  beneficial.  One  of  the 
purposes  of  this  research  was  to  explore  various  modeling 
options  and  techniques.  By  starting  with  two  different 
data  sets,  it  was  possible  to  examine  these  modeling 
options  when  used  with  both  a  more  accurate  and  a  less 
accurate  set  of  data.  This  resulted  in  a  richer  set  of 
data  comparisons  and  more  interesting  results. 

Workload  Predictors 

Several  workload  predictors  were  developed  by 
combining  output  from  the  computer  simulation  models  and 
the  basic  PRO-SWAT  predictions.  Using  the  multivariate 
comparison  of  means,  as  shown  in  Table  10,  it  appears  that 
the  most  accurate  predictions  resulted  when  Pilot  Idle  Time 
(single  server  model)  was  used  as  the  Time  Load  dimension 
surrogate.  The  Pilot  Idle  Time  predictor  (parallel  server 


model)  also  produced  reasonably  accurate  predictions,  but 
not  as  accurate  as  the  single  server  model.  The 
predictions  derived  by  using  the  Simultaneous  Task  Rate 
predictor  were  also  as  accurate  as  those  produced  by  Pilot 
Idle  Time  (single  server  model).  For  the  1986  data,  three 
predictors  (SS-TI(A),  SS-TI(B),  and  SS-TI(C))  produced 
results  that  were  significantly  different  from  the  SWAT 
data  at  the  5%  level.  These  three  predictors  also  produced 
less  accurate  predictions  for  the  1982  data.  The  last 
predictor,  SS-TI(D),  yielded  non-significant  differences, 
but  less  accurate  predictions  than  the  basic  PRO-SWAT 
predictor  for  both  the  1982  and  1986  data. 

When  the  predictors  are  evaluated  using  profile 
analysis,  as  shown  in  Tables  11  and  12,  some  of  the  same 
conclusions  can  be  drawn.  For  the  1982  data,  none  of  the 
predictors  could  be  rejected.  Some  predictors  yielded 
greater  accuracy  than  others,  but  the  results  were  mixed 
when  viewed  over  three  cockpit  configurations.  Statements 
about  the  accuracy  of  predictors  were  also  difficult 
because  the  parallel  and  coincident  hypothesis  tests  may 
show  different  trends  for  a  given  predictor.  The  1986  data 
did  have  two  predictors  (SS-TI(A)  and  SS-TI(B))  rejected  by 
the  parallel  hypothesis  test  for  the  TF/TA  configuration. 
This  was  consistent  with  the  multivariate  comparison  of 
means  results  previously  discussed.  For  the  1986  data, 


both  predictors  using  pilot  idle  time  generally  show  more 
accurate  results  in  terms  of  being  parallel  and  coincident. 
Again,  this  is  consistent  with  earlier  conclusions. 

Serial  vs  Parallel  Models 

A  fundamental  issue  addressed  in  this  research  is  the 
relative  merits  of  a  serial  processing  model  versus  a 
parallel  processing  model.  A  serial  model  is  simpler, 
easier  to  develop  and  run,  and  conceptually  easier  to 
understand.  Yet  the  serial  model  ignores  evidence  of 
actual  parallel  processing.  If  one  is  only  interested  in 
the  most  accurate  model,  which  of  these  two  models  is 
better? 

Perhaps  the  best  way  to  answer  that  is  by  comparing 
the  accuracy  of  the  two  Pilot  Idle  Time  predictors  (SS-PIR 
and  PS-PIR) .  Both  of  these  have  been  identified  as 
providing  accurate  predictions  and  they  were  computed  using 
identical  methods  in  each  model.  Therefore,  it  seems 
logical  that  differences  in  accuracy  of  these  predictors 
should  reflect  on  the  models  themselves,  rather  than  other 
factors . 

The  results  of  the  multivariate  comparison  of  means, 
as  repeated  in  Table  19,  show  the  serial  model 
predictor  produced  essentially  the  same  results  for  the 
1982  data  and  less  accurate  results  for  the  1986  data.  The 
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Table  19  Multivariate  Comparison  of  Means,  Idle  Time 
Predictor  Only 


Predictor 

PRO-SWAT 

F 

(1982) 

P  >  F 

PRO-SWAT 

F 

(1986) 

P  >  F 

PRO-SWAT 

0.33 

0.81 

1.26 

0.29 

SS-PIR 

0.32 

0.81 

2.03 

0.13 

PS-PIR 

0.44 

0.73 

2.34 

0.09 

parallel  model  produced  slightly  worse  results  for  the  1982 
data  and  worse  results  for  the  1986  data.  Ideally,  the 
better  model  should  have  produced  predictions  at  least  as 
accurate  as  the  basic  PRO-SWAT  data.  One  would  expect  that 
the  1986  PRO-SWAT  predictions  should  be  improved  by  either 
pilot  idle  time  predictor  since  the  1986  predictions  were 
not  as  accurate  as  the  1982  predictions.  In  fact,  the  1982 
predictions  were  changed  only  a  small  amount  by 
incorporating  the  pilot  idle  time  surrogate  predictor.  If 
a  choice  is  to  be  made.  Table  19  suggests  that  the  serial 
processing  model,  using  pilot  idle  time  as  a  surrogate, 
produces  slightly  more  accurate  predictions  than  the 
parallel  processing  model.  However,  it  does  not  appear 
that  this  increase  in  accuracy  is  statistically 
significant. 

Results  from  the  profile  analysis  show  no  trends 
regarding  parallel  versus  serial  models.  Table  20  repeats 
pertinent  profile  analysis  data  from  Tables  11  and  12. 


Table  20  Profile  Analysis,  Idle  Time  Predictor 


DATA 

SET 

CONFIG¬ 

URATION 

PREDIC¬ 

TOR 

PARALLEL 

H01 

COINCIDENT 

H02 

EQUAL 

H03 

1982 

ABL 

PRO-SWAT 

2.35 

0.20 

8.06 

SS-PIR 

4.63 

0.58 

10.46 

PS-PIR 

1.77 

0.02 

8.67 

TF/TA 

PRO-SWAT 

3.84 

0.42 

7.10 

SS-PIR 

1.99 

0.79 

8.71 

PS-PIR 

0.65 

1.29 

11.29 

TTA 

PRO-SWAT 

0.11 

0.01 

9.42 

SS-PIR 

0.08 

0.24 

8.66 

PS-PIR 

0.08 

0.24 

8.66 

1986 

ABL 

PRO -SWAT 

2.82 

0.02 

2.07 

SS-PIR 

2.11 

0.35 

8.17 

PS-PIR 

1.23 

0.55 

7.08 

TF/TA 

PRO-SWAT 

8.07 

0.04 

0.94 

SS-PIR 

5.50 

0.13 

1.84 

PS-PIR 

5.83 

0.76 

4.08 

TTA 

PRO-SWAT 

3.66 

0.14 

2.45 

SS-PIR 

1.62 

0.04 

6.89 

PS-PIR 

1.62 

0.04 

6.89 

Examination  of  this  data  shows  no  evidence  that  either 
model  is  more  accurate  over  both  data  sets  and  all  three 
flight  configurations  in  terms  of  being  both  parallel  and 
coincident. 

The  general  conclusion  to  be  drawn  from  the  two 
analysis  methods  just  described  is  that  the  serial 
processing  model  is  perhaps,  slightly  more  accurate. 
However,  this  should  be  viewed  with  caution  because  the 
results  were  not  totally  conclusive  and  probably  not 


statistically  significant.  The  multivariate  comparison  of 
means  suggested  that  the  serial  processing  model  was 
slightly  more  accurate.  However,  the  profile  analysis 
shows  mixed  results  and  no  clear  trend.  Based  on  this 
evidence,  no  strong  statement  can  be  made  suggesting  that 
either  model  consistently  produces  more  accurate  results. 


Expert  versus  Non-expert  Pilot  Subjects 

Two  predictor  data  sets  were  used  in  this  research. 

The  1982  set  was  based  on  workload  estimates  made  by  F-15 
pilots  predicting  workload  in  an  advanced  version  of  the  F- 
15.  The  1986  set  of  predictions  were  made  by  A-7  pilots 
estimating  workload  for  the  same  advanced  F-1S  cockpit.  It 
is  reasonable  to  question  the  wisdom  of  using  A-7  pilots 
for  this  task  since  they  were  less  than  experts  in  the  F- 
15. 


To  address  this,  several  issues  must  be  raised.  The 
mission  scenarios  evaluated  in  the  RAN/ACE  experiments  were 
all  air-to-oround  missions.  As  the  F-15  currently  exists, 
it  is  exclusively  an  air-to-air  mission  aircraft.  This 
implies  that  all  F-15  pilots  used  in  the  1982  experiment 
were  trained  in  the  air-to-air  mission,  not  the  air-to- 
ground  mission  being  evaluated.  While  the  F-15  pilots  may 
have  been  knowledgeable  in  basic  F-15  systems,  they  were 
not  especially  knowledgeable  in  the  air-to-ground  mission. 


On  the  other  hand,  the  A-7  pilots  used  in  the  1986  RAM/ACE 
experiment  replication  were  skilled  in  the  air-to-ground 
mission,  since  that  is  the  primary  mission  of  the  A-7 
aircraft.  As  stated  previously,  the  A-7  pilots  were  not 
skilled  in  the  F-15  aircraft. 

It  is  not  at  all  clear  which  factor  was  more 
important:  knowledge  of  the  F-15  or  knowledge  of  the  air- 
to-ground  mission.  One  might  expect  the  F-15  pilots  to 
provide  more  accurate  predictions,  but  there  is  no  way  to 
prove  this  assumption  in  this  research.  Not  only  were 
there  experience  differences  in  the  pilots  used  in  both 
experiments,  but  there  were  also  differences  in  the  way  the 
experiments  were  administered.  Much  tighter  experimental 
control  was  possible  in  the  1982  experiment,  as  described 
in  Chapter  III,  than  in  the  1986  replication.  Lack  of 
tight  control  in  this  replication  may  have  contributed  to 
the  relative  inaccuracy  of  the  1986  data.  However,  it  is 
not  possible  to  separate  inaccuracies  resulting  from  lack 
of  experimental  control  and  those  resulting  from  lack  of  F- 
15  expertise. 

In  summary,  the  pilots  used  to  develop  each  predictor 
data  set  were  experts,  but  in  different  facets  of  the 
mission  scenario  being  evaluated.  Ideally,  expert  pilots 
in  both  facets  should  have  been  used,  but  this  was  not 
possible.  It  is  not  clear  that  F-15  experience  was  more 
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important  than  air-to-ground  experience  since  different 
levels  of  experimental  control  were  used  in  each  case. 
However,  it  is  clear  that  the  1982  predictions  were  more 
accurate  than  the  1986  predictions  and  this  did  have  an 
impact  on  the  accuracy  of  all  derived  predictors. 

Sensitivity  Issues 

Chapter  V  examined  several  assumptions  and  evaluated 
the  sensitivity  of  the  models  to  these  assumptions.  The 
first  considered  four  alternative  definitions  for  task 
interruption  rate.  Both  the  multivariate  comparison  of 
means  and  the  profile  analysis  recognized  problems  with 
predictors  A  and  B.  This  suggests  that  task  preemption  is 
not  an  adequate  definition  for  a  task  interruption.  Of  the 
two  remaining  predictors,  both  based  on  the  arrival  of  a 
new  task  when  at  least  one  other  task  is  already  being 
worked,  only  one  (D)  showed  non-significant  differences 
from  the  SWAT  data.  This  occurred  when  this  latter 
definition  of  task  interrupts  was  used  and  subsequent 
arrivals  of  preempted  tasks  are  not  counted  as  new 
arrivals.  It  should  be  pointed  out,  however,  that  all 
predictors  based  on  task  interrupt  rate  yielded  predictions 
less  accurate  than  the  basic  PRO-SWAT  data  or  the 
predictors  based  on  pilot  idle  time  or  simultaneous  task 


rate. 
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Another  assumption  examined  was  the  decision  to  ignore 
workload  experts'  significantly  different  estimate  of  one 
of  the  transformation  limits.  Tables  13  and  14  show  that 
using  the  experts'  transformation  yielded  less  accurate 
results  than  those  provided  by  the  original  transformation. 
Given  the  inherent  limits  of  this  transformation,  it  is 
acceptable  to  ignore  this  expert  opinion  and  accept  the 
combined  inputs  of  all  subjects  as  defining  the 
transformation  most  accurately. 

The  third  assumption  examined  in  Chapter  V  was  the 
arbitrary  decision  to  use  1.0  as  the  degradation  limit  for 
increasing  service  times.  This  only  applied  to  the 
parallel  processing  model  when  a  new  task  was  being 
considered  for  parallel  processing.  The  sensitivity  of 
this  assumption  was  examined  and  it  was  found  that  the 
simultaneous  task  rate  predictor  (PS-STR)  was  sensitive  to 
this  limit,  while  the  pilot  idle  time  predictor  was 
relatively  insensitive.  When  predictor  accuracy  was 
determined  for  several  levels  of  the  degradation  limit, 
results  were  mixed  using  both  profile  analysis  and 
multivariate  comparison  of  means,  as  shown  in  Tables  16, 

17,  and  18. 

Perhaps  the  definition  of  simultaneous  task  rate  as 
used  in  this  research  is  the  reason  for  these  mixed 
results.  This  rate  was  computed  as  the  percent  of  time  a 
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pilot  was  occupied  with  two  or  more  tasks  simultaneously. 

As  defined,  it  ignored  differences  between  working  two 
tasks  simultaneously  and  three  tasks  simultaneously.  One 
would  expect  higher  workload  for  the  latter  case  than  for 
the  former  case.  This  definition  will  not  recognize  this. 
In  a  sense,  the  simultaneous  task  rate  definition  is  the 
complement  to  pilot  idle  rate,  if  working  on  a  single  task 
is  viewed  as  being  "idle".  This  suggests  that  this 
definition  of  simultaneous  task  rate  was,  in  a  sense, 
measuring  almost  the  same  underlying  concept  as  pilot  idle 
rate.  Since  pilot  idle  rate  was  found  to  be  a  relatively 
accurate  predictor,  changing  the  definition  slightly  will 
most  likely  result  in  a  similar  level  of  accuracy.  This 
conclusion  is  consistent  with  the  results. 

General  Conclusion 

The  previous  sections  discussed  the  accuracy  of  this 
model  when  predicting  pilot  workload  in  a  specific 
situation.  The  evidence  cited  generally  supports  the 
validity  of  this  modeling  approach.  Of  all  the  predictors 
evaluated,  it  appears  that  the  Pilot  Idle  Time  predictors 
and  the  Simultaneous  Task  Rate  predictor  produce  the  most 
accurate  results.  As  described  in  the  previous  section, 
simultaneous  task  rate  may  be  measuring  the  same  underlying 
concept  as  pilot  idle  rate.  Therefore  it  is  not  surprising 


that  these  three  predictors  should  produce  similar  results. 
As  good  as  these  results  were,  they  should  be  viewed  only 
as  preliminary  indicators  of  the  technique's  validity. 
Further  research  should  be  conducted  to  increase  confidence 
in  this  modeling  methodology. 

Recommendations 

At  the  completion  of  this  research,  several 
suggestions  can  be  made  regarding  ways  in  which  this 
modeling  methodology  can  be  improved  or  further 
investigated.  The  first  and  most  obvious  would  be  to 
reaccomplish  the  predictions  against  another  set  of 
workload  measurements.  By  comparing  predictor  accuracy 
against  an  independent  set  of  measurements,  it  would  be 
possible  to  further  validate  the  model.  If  this  is  done, 
care  should  be  taken  to  insure  that  a  large  number  of  data 
samples  are  taken.  This  should  provide  more  discrimination 
between  predictors  and  greater  confidence  in  the  results. 

The  transformation  developed  in  this  research  to 
convert  simulation  output  to  workload  measurements  was 
adequate,  but  could  br*  improved.  As  mentioned  previously, 
the  survey  used  transformed  in  the  reverse  direction  and 
assumed  that  this  transformation  was  reversible.  Since  the 
transformation  was  based  on  a  survey,  it  is  subject  to  all 
the  problems  of  a  survey.  For  example,  statements  and 


questions  can  be  misunderstood,  wrong  answers  recorded,  or 
unintentional  biases  included.  Furthermore,  opinions 
recorded  as  numerical  ratings  may  not  correspond  to  "true1' 
results.  A  possible  improvement  would  be  to  conduct  an 
experiment  in  a  flight  simulator.  SWAT  workload 
measurements  could  be  taken  and  some  method  developed  to 
assess  pilot  idle  time,  task  interruption  rate,  and 
simultaneous  task  rate.  With  these  data  points  recorded 
over  a  large  number  of  pilots,  a  more  direct  transformation 
could  be  made. 

The  degradation  factors  used  in  the  parallel 
processing  model  apparently  worked  well  enough.  The 
factors  were  determined  by  computing  the  number  of  resource 
overlaps  using  the  multiple  resource  model.  This  is  a 
modeling  simplification  because  it  only  counts  overlaps  and 
ignores  the  types  of  resource  overlaps  occurring.  A 
possible  model  refinement  would  be  to  consider  overlap 
types  and  combinations  of  resources  demanded.  This  may 
result  in  more  accurate  predictions. 

It  may  also  be  possible  to  convert  directly  from  the 
resources  used,  as  defined  by  Wickens'  multiple  resource 
model  (44,  46,  and  47),  to  SWAT  scores  without  going 
through  a  surrogate  predictor  and  transformation.  If  the 
multiple  resource  model  is  a  reasonably  accurate 
representation  of  the  fundamental  capacities  of  a  pilot  and 


the  demands  on  those  capacities,  it  should  also  recognize 
varying  levels  of  pilot  workload.  Workload  is  a  construct 
that  describes  the  demand  on  fundamental  pilot  capacities. 
If  the  multiple  resource  model  reflects  these  capacities, 
it  should  also  reflect  demand  on  these  capacities,  or 
workload.  To  derive  SWAT  workload  measures,  it  would  be 
necessary  to  convert  task  demands  on  the  2x3  multiple 
resource  model  matrix  to  SWAT  scores.  If  the  assumption  is 
made  that  the  multiple  resource  model  only  reflects  the 
Time  Load  dimension  of  the  SWAT  methodology,  a  method  to 
convert  resource  demands  to  LOW,  MEDIUM,  and  HIGH  ratings 
must  be  developed.  Perhaps  this  could  be  accomplished 
through  another  flight  simulator  experiment. 

The  final  recommendation  concerns  the  possibility  of 
using  the  existing  predictors  in  another  way.  Instead  of 
evaluating  each  predictor  individually,  it  may  be  possible 
to  find  some  combination  of  predictors  which  produces  more 
accurate  predictions.  The  SWAT  methodology  assumes  that 
workload  is  characterized  by  pilot  idle  rate,  task 
interruption  rate,  and  simultaneous  task  rate.  Perhaps 
some  combination  of  these  three  predictors  could  provide 
more  accurate  predictions. 
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Contributions 

At  the  conclusion  of  this  research,  it  is  possible  to 
identify  specific  contributions  as  follows: 

-  Two  simulation  models  have  been  developed  which 
emulate  a  pilot  working  on  certain  tasks  in  a  cockpit. 

From  these  models  it  is  possible  to  determine  surrogate 
workload  predictors.  The  validity  of  these  models  as 
predictors  of  the  Time  Load  dimension  of  the  SWAT 
methodology  has  been  supported. 

-  Various  definitions  of  Time  Load  surrogate 
predictors  have  been  investigated  and  the  most  promising 
have  been  identified. 

-  The  parallel  processing  model  developed 
incorporates  Wickens1  multiple  resource  model  to  determine 
pilot  resources  available  to  accomplish  a  task.  This  has 
not  been  done  before  and  only  recently  was  suggested  by 
Wickens'  (48). 

-  Proper  use  of  this  workload  model  could  provide  a 
tool  for  accurately  estimating  pilot  workload.  A  potential 
exists  to  extend  this  to  other  domains.  Furthermore,  the 
model  could  be  used  to  cross-check  or  validate  other 
workload  prediction  techniques. 

-  Areas  for  possible  future  research  have  been 
identified  and  suggestions  made  toward  initiating  that 
research. 


APPENDIX  A 


SWAT  CARDS  AND  SORTING  INSTRUCTIONS 
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This  appendix  contains  a  copy  of  the  SWAT  Card  Sort 
Instructions  given  to  each  subject  before  the  SWAT  card 
sort  was  begun.  This  card  sort  was  completed  prior  to 
making  any  SWAT  estimates  and  was  used  to  establish  a 
unique,  linear  scale  for  each  subject.  The  exact 
procedures  used  are  described  in  Chapter  III. 


The  instructions  begin  on  the  next  page  and  these  are 


SWAT  CARD  SORT  INSTRUCTIONS  FOR  SUBJECTS 

During  the  course  of  this  experiment,  you  will  be 
asked  to  quantify  the  mental  workload  required  to  complete 
the  tasks  you  will  be  performing.  Mental  Workload  refers 
to  how  hard  you  work  to  accomplish  some  task,  groups  of 
tasks,  or  an  entire  job.  The  workload  imposed  on  you  at 
any  one  time  consists  of  a  combination  of  various 
dimensions  which  contribute  to  the  subjective  feeling  of 
workload.  The  Subjective  Workload  Assessment  Technique 
(SWAT)  defines  these  dimensions  as  (1)  Time  Load,  (2) 
Mental  Effort  Load,  and  (3)  Psychological  Suress  Load. 

For  the  purpose  of  SWAT,  the  three  dimensions  have 
been  assigned  three  levels.  The  dimensions  and  their 
levels  are  described  in  the  following  paragraphs. 

Time  Load 

Time  Load  refers  to  the  fraction  of  the  total  time 
that  you  are  busy.  When  time  load  is  low,  sufficient  time 
is  available  to  complete  ail  of  your  mental  work  with  some 
time  to  spare.  As  time  load  increases,  spare  time  drops 
out  and  some  aspects  of  performance  overlap  and  interrupt 
one  another.  This  overlap  and  interruption  can  come  from 
performing  more  than  one  task  or  from  different  aspects  of 
performing  the  same  task.  At  higher  levels  of  time  load, 
several  aspects  of  performance  often  occur  simultaneously, 
you  are  busy,  and  interruptions  are  very  frequent. 


Tine  load  may  be  rated  on  the  three  point  scale  below. 


1.  Often  have  spare  time.  Interruptions  or  overlap 
among  activities  occur  infrequently  or  not  at  all. 

2.  Occasionally  have  spare  time.  Interruptions  or 
overlap  among  activities  are  very  frequent. 

3.  Almost  never  have  spare  time.  Interruptions  or 
overlap  amo.ig  activities  are  very  frequent,  or  occur  all 
the  time. 

Mental  Effort  Load 

As  described  above,  time  load  refers  to  the  amount  of 
time  one  has  available  to  perform  a  task  or  tasks.  In 
contrast,  mental  effort  load  is  an  index  of  the  amount  of 
attention  or  mental  effort  required  by  a  task  regardless  of 
the  number  of  tasks  to  be  performed  or  any  time 
limitations.  When  mental  effort  load  is  low,  the 
concentration  and  attention  required  by  a  task  is  minimal 
and  performance  is  nearly  automatic.  As  the  demand  for 
mental  effort  increases,  due  to  task  complexity  or  the 
amount  of  information  which  must  be  dealt  with  in  order  to 
perform  adequately,  the  degree  of  concentration  and 
attention  required  increases.  High  mental  effort  load 
demands  total  attention  or  concentration  due  to  task 
complexity  or  the  amount  of  information  that  must  be 
processed . 


Mental  effort  load  may  be  rated  using  the  three  point 
scale  below. 

1.  Very  little  conscious  mental  effort  or 
concentration  required.  Activity  is  almost  automatic, 
requiring  little  or  no  attention. 

2.  Moderate  conscious  mental  effort  or  concentration 
required.  Complexity  of  activity  is  moderately  high  due  to 
uncertainty,  unpredictability,  or  unfamiliarity. 
Considerable  attention  required. 

3.  Extensive  mental  effort  and  concentration  are 
necessary.  Very  complex  activity  requiring  total 
attention. 

Psychological  Stress  Load 

Stress  load  refers  to  the  contribution  to  total 
workload  of  any  conditions  that  produce  anxiety, 
frustration,  or  confusion  while  performing  a  task  or  tasks. 
At  low  levels  of  stress,  one  feels  relatively  relaxed.  As 
stress  increases,  confusion,  anxiety,  or  frustration 
increase  and  greater  concentration  and  determination  are 
required  to  maintain  control  of  the  situation. 

Psychological  stress  load  may  be  rated  on  the  three 
point  scale  below. 

1.  Little  confusion,  risk,  frustration,  or  anxiety 
exists  and  can  be  easily  accommodated. 


2.  Moderate  stress  due  to  confusion,  frustration,  or 
anxiety  noticeably  adds  to  workload.  Significant 
compensation  is  required  to  maintain  adequate  performance. 

3.  High  to  very  intense  stress  due  to  confusion, 
frustration,  or  anxiety.  High  to  extreme  determination  and 
self-control  required. 

Each  of  the  three  dimensions  just  described  contribute 
to  workload  during  performance  of  a  task  or  group  of  tasks. 
Note  that  although  all  three  'actors  may  be  correlated, 
they  need  not  be.  For  example,  one  can  have  many  tasks  to 
perform  in  the  time  available  (high  time  load)  but  the 
tasks  may  require  little  concentration  (low  mental  effort) . 
Likewise,  one  can  be  anxious  and  frustrated  (high  stress) 
and  have  plenty  of  spare  time  between  relatively  simple 
tasks.  Since  the  three  dimensions  contributing  to  workload 
are  not  necessarily  correlated,  please  treat  each  dimension 
individually  and  give  independent  assessments  of  the  time 
load,  mental  effort  load,  and  stress  load  that  you 
experience  in  performing  the  following  tasks. 

One  of  the  most  important  features  of  SWAT  is  its 
unique  scoring  system.  SWAT  uses  a  procedure  to  find 
separate  scoring  weights  for  each  level  of  the  dimension. 
Then,  it  determines  a  distinctive  workload  scale  for  each 
person.  The  scaling  system  greatly  improves  the  precision 
of  the  workload  ratings  you  will  give  later. 


In  order  to  develop  your  individual  scale,  we  need 
information  from  you  regarding  the  amount  of  workload  you 
feel  is  imposed  by  various  combinations  of  the  dimensions 
described  above.  We  get  this  information  by  having  you 
rank  order  the  workload  associated  with  each  of  the 
combinations. 

In  order  for  you  to  rank  order  the  workload  for  each 
of  the  combinations,  you  have  been  given  a  set  of  27  cards 
with  the  combinations  from  each  of  the  three  dimensions. 
Each  card  contains  a  different  combination  of  levels  of 
Time  Load,  Mental  Effort,  and  Psychological  Stress.  Your 
job  is  to  sort  the  cards  so  that  they  are  rank  ordered 
according  to  the  level  of  workload  represented  on  each. 

In  completing  your  card  sorts,  please  consider  the 
workload  imposed  on  a  person  by  the  combination  represented 
in  each  card.  Arrange  the  cards  from  the  lowest  workload 
condition  through  the  highest  condition.  You  may  use  any 
strategy  you  choose  in  rank  ordering  the  cards.  One 
strategy  that  proves  useful  is  to  arrange  the  cards  into  a 
number  of  preliminary  stacks  representing  "High", 
"Moderate",  and  "Low"  workload.  Individual  cards  can  be 
exchanged  between  stacks,  if  necessary,  and  then  rank 
ordered  within  stacks,  stacks  can  then  be  recombined  and 
checked  to  be  sure  that  they  represent  your  ranking  of 
lowest  to  highest  workload.  However,  the  choice  of 
strategy  is  up  to  you  and  you  should  choose  the  one  that 


works  best  for  you. 

There  is  no  "school  solution"  to  this  problem.  There 
is  no  correct  order.  The  correct  order  is  what,  in  your 
judgment,  best  describes  the  progression  of  workload  from 
lowest  to  highest  for  a  general  case  rather  than  any 
specific  event.  That  judgment  differs  for  each  of  us.  The 
letters  you  see  on  the  back  of  the  cards  are  to  allow  us  to 
arrange  the  cards  in  a  previously  randomized  sequence  so 
that  everyone  gets  the  same  order.  If  you  examine  your 
deck  you  will  see  the  order  on  the  back  runs  from  A  through 
Z  and  then  ZZ. 

Please  remember: 

1.  The  card  sort  is  being  done  so  a  workload  scale 
may  be  developed  for  you.  This  scale  will  have  a  distinct 
workload  value  for  each  possible  combination  of  Time  Load, 
Mental  Effort  Load,  and  Psychological  Stress  Load. 

Time _ Effort _ Stress _ Workload  Scale 
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33  3  100 

2.  When  performing  the  card  sorts,  use  the 
descriptors  printed  on  the  cards.  Please  remember  not  to 


sort  the  cards  based  on  a  particular  task  (such  as  flying 
an  airplane) .  Sort  the  cards  according  to  your  general 
view  of  workload  and  how  important  you  consider  the 
dimensions  of  time,  mental  effort,  and  psychological  stress 
load  to  be. 

3.  During  the  actual  experiment,  you  will  read  a 
description  of  the  desired  tasks.  Then,  you  will  provide  a 
SWAT  score  based  on  your  opinion  of  the  mental  workload 
required  to  perform  the  task.  This  SWAT  score  will  consist 
of  one  number  from  each  of  the  three  dimensions.  For 
example,  a  possible  SWAT  score  is  1  -  2  -  2.  This 
represents  a  1  for  Time  Load,  a  2  for  Mental  Effort  Load, 
and  a  2  for  Psychological  Stress  Load. 

4.  We  are  not  asking  for  your  preference  concerning 
Time,  Mental  Effort,  and  Psychological  Stress  Load.  Some 
people  may  prefer  to  be  "busy"  rather  than  "idle1'  in  either 
the  Time  Load,  Mental  Effort  Load  or  Psychological  stress 
Load  dimension.  We  are  not  concerned  with  this  preference. 
We  need  information  on  how  the  three  dimensions  and  the 
three  levels  of  each  one  will  affect  the  level  of  workload 
as  you  see  it.  You  may  prefer  a  2  -  2  -  2  situation 
instead  of  a  1  -  1  -  1  situation.  But,  you  should  still 
realize  that  the  1  -  l  -  1  situation  imposes  less  workload 
on  you  and  leaves  a  greater  reserve  capacity. 


From  this  point  until  you  have  completed  the  sorting 


will  probably  take  30  minutes  to  an  hour.  Please  feel  free 
to  ask  questions  at  any  time.  Thank  you  for  your 
cooperation. 


Card  A 


Almost  never  have  spare  time.  Interruptions  or 
overlap  among  activities  are  very  frequent,  or  occur  all 
the  time. 

Moderate  conscious  mental  effort  or  concentration 
required.  Complexity  of  activity  is  moderately  high  due  to 
uncertainty,  unpredictability,  or  unfamiliarity. 
Considerable  attention  required. 

Moderate  stress  due  to  confusion,  frustration,  or 
anxiety  noticeably  adds  to  workload.  Significant 
compensation  is  required  to  maintain  adequate  performance. 


Card  B 

Often  have  spare  time.  Interruptions  or  overlap  among 
activities  occur  infrequently  or  not  at  all. 

Very  little  conscious  mental  effort  or  concentration 
required.  Activity  is  almost  automatic,  requiring  little 
or  no  attention. 

Moderate  stress  due  to  confusion,  frustration,  or 
anxiety  noticeably  adds  to  workload.  Significant 
compensation  is  required  to  maintain  adequate  performance. 


Card  C 

Often  have  spare  time.  Interruptions  or  overlap  among 
activities  occur  infrequently  or  not  at  all. 

Moderate  conscious  mental  effort  or  concentration 
required.  Complexity  of  activity  is  moderately  high  due  to 
uncertainty,  unpredictability,  or  unfamiliarity. 
Considerable  attention  required. 

High  to  very  intense  stress  due  to  confusion, 
frustration,  or  anxiety.  High  to  extreme  determination  and 
self-control  required. 
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Card  D 

Almost  never  have  spare  time.  Interruptions  or 
overlap  among  activities  are  very  frequent,  or  occur  all 
the  time. 

Very  little  conscious  mental  effort  or  concentration 
required.  Activity  is  almost  automatic,  requiring  little 
or  no  attention. 

High  to  very  intense  stress  due  to  confusion, 
frustration,  or  anxiety.  High  to  extreme  determination  and 
self-control  required. 


Card  E 


Occasionally  have  spare  time.  Interruptions  or 
overlap  among  activities  are  very  frequent. 

Extensive  mental  effort  and  concentration  are 
necessary.  Very  complex  activity  requiring  total 
attention. 

Moderate  stress  due  to  confusion,  frustration,  or 
anxiety  noticeably  adds  to  workload.  Significant 
compensation  is  required  to  maintain  adequate  performance. 


Card  F 


Often  have  spare  time.  Interruptions  or  overlap  among 
activities  occur  infrequently  or  not  at  all. 

Moderate  conscious  mental  effort  or  concentration 
required.  Complexity  of  activity  is  moderately  high  due  to 
uncertainty,  unpredictability,  or  unfamiliarity. 
Considerable  attention  required. 

Little  confusion,  risk,  frustration,  or  anxiety  exists 
and  can  be  easily  accommodated. 
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Card  G 


Occasionally  have  spare  time.  Interruptions  or 
overlap  among  activities  are  very  frequent. 

Very  little  conscious  mental  effort  or  concentration 
required.  Activity  is  almost  automatic,  requiring  little 
or  no  attention. 

Moderate  stress  due  to  confusion,  frustration,  or 
anxiety  noticeably  adds  to  workload.  Significant 
compensation  is  required  to  maintain  adequate  performance. 


Card  H 


Almost  never  have  spare  time.  Interruptions  or 
overlap  among  activities  are  very  frequent,  or  occur  all 
the  time. 

Very  little  conscious  mental  effort  or  concentration 
required.  Activity  is  almost  automatic,  requiring  little 
or  no  attention. 

Little  confusion,  risk,  frustration,  or  anxiety  exists 
and  can  be  easily  accommodated. 


Card  I 


Almost  never  have  spare  time.  Interruptions  or 
overlap  among  activities  are  very  frequent,  or  occur  all 
the  time. 

Extensive  mental  effort  and  concentration  are 
necessary.  Very  complex  activity  requiring  total 
attention. 

High  to  very  intense  stress  due  to  confusion, 
frustration,  or  anxiety.  High  to  extreme  determination  and 
self-control  required. 


Card  J 


Often  have  spare  time.  Interruptions  or  overlap  among 
activities  occur  infrequently  or  not  at  all. 

Moderate  conscious  mental  effort  or  concentration 
required.  Complexity  of  activity  is  moderately  high  due  to 
uncertainty,  unpredictability,  or  unfarailiarity. 
Considerable  attention  required. 

Moderate  stress  due  to  confusion,  frustration,  or 
anxiety  noticeably  adds  to  workload.  Significant 
compensation  is  required  to  maintain  adequate  performance. 


Card  K 

Occasionally  have  spare  time.  Interruptions  or 
overlap  among  activities  are  very  frequent. 

Extensive  mental  effort  and  concentration  are 
necessary.  Very  complex  activity  requiring  total 
attention. 

Little  confusion,  risk,  frustration,  or  anxiety  exists 
and  can  be  easily  accommodated. 


Card  L 

Almost  never  have  spare  time.  Interruptions  or 
overlap  among  activities  are  very  frequent,  or  occur  all 
the  time. 

Extensive  mental  effort  and  concentration  are 
necessary.  Very  complex  activity  requiring  total 
attention. 

Little  confusion,  risk,  frustration,  or  anxiety  exists 
and  can  be  easily  accommodated. 


Card  M 


Often  have  spare  time.  Interruptions  or  overlap  among 
activities  occur  infrequently  or  not  at  all. 

Extensive  mental  effort  and  concentration  are 
necessary.  Very  complex  activity  requiring  total 
attention. 

High  to  very  intense  stress  due  to  confusion, 
frustration,  or  anxiety.  High  to  extreme  determination  and 
self-control  required. 


Card  N 

Often  have  spare  time.  Interruptions  or  overlap  among 
activities  occur  infrequently  or  not  at  all. 

Very  little  conscious  mental  effort  or  concentration 
required.  Activity  is  almost  automatic,  requiring  little 
or  no  attention. 

Little  confusion,  risk,  frustration,  or  anxiety  exists 
and  can  be  easily  accommodated . 


Card  0 

Almost  never  have  spare  time.  Interruptions  or 
overlap  among  activities  are  very  frequent,  or  occur  all 
the  time. 

Moderate  conscious  mental  effort  or  concentration 
required.  Complexity  of  activity  is  moderately  high  due  to 
uncertainty,  unpredictability,  or  unfamiliarity. 
Considerable  attention  required. 

High  to  very  intense  stress  due  to  confusion, 
frustration,  or  anxiety.  High  to  extreme  determination  and 
self-control  required. 


Card  P 


Almost  never  have  spare  time.  Interruptions  or 
overlap  among  activities  are  very  frequent,  or  occur  all 
the  time. 

Very  little  conscious  mental  effort  or  concentration 
required.  Activity  is  almost  automatic,  requiring  little 
or  no  attention. 

Moderate  stress  due  to  confusion,  frustration,  or 
anxiety  noticeably  adds  to  workload.  Significant 
compensation  is  required  to  maintain  adequate  performance. 


Card  Q 

Occasionally  have  spare  time.  Interruptions  or 
overlap  among  activities  are  very  frequent. 

Moderate  conscious  mental  effort  or  concentration 
required.  Complexity  of  activity  is  moderately  high  due  to 
uncertainty,  unpredictability,  or  unfamiliarity. 
Considerable  attention  required. 

Moderate  stress  due  to  confusion,  frustration,  or 
anxiety  noticeably  adds  to  workload.  Significant 
compensation  is  required  to  maintain  adequate  performance. 


Card  R 

Occasionally  have  spare  time.  Interruptions  or 
overlap  among  activities  are  very  frequent. 

Extensive  mental  effort  and  concentration  are 
necessary.  Very  complex  activity  requiring  total 
attention. 

High  to  very  intense  stress  due  to  confusion, 
frustration,  or  anxiety.  High  to  extreme  determination  and 
self-control  required. 


Card  S 


Often  have  spare  time.  Interruptions  or  overlap  among 
activities  occur  infrequently  or  not  at  all. 

Extensive  mental  effort  and  concentration  are 
necessary.  Very  complex  activity  requiring  total 
attention. 

Moderate  stress  due  to  confusion,  frustration,  or 
anxiety  noticeably  adds  to  workload.  Significant 
compensation  is  required  to  maintain  adequate  performance. 


Card  T 

Almost  never  have  spare  time.  Interruptions  or 
overlap  among  activities  are  very  frequent,  or  occur  all 
the  time. 

Extensive  mental  effort  and  concentration  are 
necessary.  Very  complex  activity  requiring  total 
attention. 

Moderate  stress  due  to  confusion,  frustration,  or 
anxiety  noticeably  adds  to  workload,  significant 
compensation  is  required  to  maintain  adequate  performance. 


Card  U 

Occasionally  have  spare  time.  Interruptions  or 
overlap  among  activities  are  very  frequent. 

Very  little  conscious  mental  effort  or  concentration 
required.  Activity  is  almost  automatic,  requiring  little 
or  no  attention. 

Little  confusion,  risk,  frustration,  or  anxiety  exists 
and  can  be  easily  accommodated. 
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Card  V 

Occasionally  have  spare  time.  Interruptions  or 
overlap  among  activities  are  very  frequent. 

Moderate  conscious  mental  effort  or  concentration 
required.  Complexity  of  activity  is  moderately  high  due  to 
uncertainty,  unpredictability,  or  unfamiliarity. 
Considerable  attention  required. 

Little  confusion,  risk,  frustration,  or  anxiety  exists 
and  can  be  easily  accommodated. 


Card  w 

Often  have  spare  time.  Interruptions  or  overlap  among 
activities  occur  infrequently  or  not  at  all. 

Very  little  conscious  mental  effort  or  concentration 
required.  Activity  is  almost  automatic,  requiring  little 
or  no  attention. 

High  to  very  intense  stress  due  to  confusion, 
frustration,  or  anxiety.  High  to  extreme  determination  and 
self-control  required. 


Card  X 

Often  have  spare  time.  Interruptions  or  overlap  among 
activities  occur  infrequently  or  not  at  all. 

Extensive  mental  effort  and  concentration  are 
necessary.  Very  complex  activity  requiring  total 
attention. 

Little  confusion,  risk,  frustration,  or  anxiety  exists 
and  can  be  easily  accommodated. 
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Card  Y 


Almost  never  have  spare  time.  Ir*"  ruptions  or 
overlap  among  activities  are  very  fre-  .ent,  or  occur  all 
the  time. 

Moderate  conscious  mental  effort  or  concentration 
required.  Complexity  of  activity  is  moderately  high  due  to 
uncertainty,  unpredictability,  or  unfamiliarity. 
Considerable  attention  required. 

Little  confusion,  risk,  frustration,  or  anxiety  exists 
and  can  be  easily  accommodated. 


Card  Z 


Occasionally  have  spare  time.  Interruptions  or 
overlap  among  activities  are  very  frequent. 


Very  little  conscious  mental  effort  or  concentration 
required.  Activity  is  almost  automatic,  requiring  little 
or  no  attention. 

High  to  very  intense  stress  due  to  confusion, 
frustration,  or  anxiety.  High  to  extreme  determination  and 
self-control  required. 


Card  ZZ 

Occasionally  have  spare  time.  Interruptions  or 
overlap  among  activities  are  very  frequent. 

Moderate  conscious  mental  effort  or  concentration 
required.  Complexity  of  activity  is  moderately  high  due  to 
uncertainty,  unpredictability,  or  unfamiliarity. 
Considerable  attention  required. 

High  to  very  intense  stress  due  to  confusion, 
frustration,  or  anxiety.  High  to  extreme  determination  and 
self-control  required. 


APPENDIX  B 


SWAT  CARD  SORT  RESULTS 


A  total  of  thirteen  subjects  participated  in  the 
workload  prediction  experiment  conducted  by  the  author. 

All  subjects  were  experienced  Air  Force  pilots.  The  first 
eight  flew  the  single-seat  A-7  aircraft  and  the  remaining 
five  flew  the  two-seat  F-4  aircraft.  After  receiving 
initial  instructions,  each  pilot  was  asked  to  review  the  27 
cards  shown  in  Appendix  A  and  arrange  them  in  ascending 
order  from  lowest  workload  to  highest  workload.  The 
results  of  this  sorting  process  are  shown  in  Table  21. 

This  table  shows  the  ordered  position  for  each  card,  from  1 
to  27,  by  subject,  for  each  possible  combination  of  Time 
Load,  Mental  Effort  Load,  and  Psychological  Stress  Load. 

This  data  was  then  processed  by  a  conjoint 
measurement,  computer-implemented  algorithm  to  translate 
the  raw  data  to  an  ordinal  scale.  This  algorithm  also 
identified  the  dominant  group  to  which  each  subject 
belonged.  That  is,  experience  has  shown  that  individual 
subjects  typically  view  one  dimension  (time,  mental  effort, 
or  psychological  stress)  as  dominant.  The  algorithm 
determines  this  and  computes  a  scale  for  each  dominant 
group  as  well  as  a  scale  for  the  entire  group.  These 
scales  are  shown  in  Tables  22  through  25.  These  scales  are 
used  to  convey  subsequent  low,  medium,  and  high  ratings 
(i.e.,  1,  2,  3)  in  the  three  SWAT  dimensions  to  an  ordinal 
scale.  These  linearized  scores  are  then  used  as  estimates 


of  pilot  workload 
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Table  21  CARD  SORT  DATA 


(T,M, P)  ORIGINAL  SUBJECT 

ORDER  I  1  2  3  4  5  6  7  8  9  10  11  12  13 


(1,1,1) 

(1,1,2) 

(1.1.3) 
(1,2/1) 
(1/2,2) 

(1.2.3) 
d/3,1) 

(1.3.2) 

(1.3.3) 

(2,1,1) 

(2.1.3) 

(2.1.3) 
(2,2,1) 
(2,2,2) 

(2.2.3) 

(2.3.1) 

(2.3.2) 

(2.3.3) 

(3.1.1) 

(3.1.2) 

(3.1.3) 

(3.2.1) 

(3.2.2) 

(3.2.3) 

(3.3.1) 

(3.3.2) 

(3.3.3) 


(T,M, P) 


1 

1 

1 

1 

1 

1 

1 

5 

3 

4 

5 

3 

16 

7 

3 

4 

5 

11 

6 

15 

16 

2 

2 

8 

3 

2 

12 

2 

4 

5 

11 

9 

7 

10 

9 

5 

6 

21 

18 

12 

23 

18 

8 

7 

16 

6 

5 

5 

5 

7 

8 

18 

15 

11 

21 

11 

6 

9 

22 

24 

18 

25 

20 

2 

10 

2 

2 

4 

2 

3 

8 

12 

6 

8 

9 

7 

8 

4 

11 

7 

16 

16 

14 

17 

6 

13 

9 

7 

8 

8 

4 

9 

14 

12 

13 

17 

11 

10 

7 

15 

15 

23 

23 

19 

19 

9 

16 

19 

12 

13 

4 

6 

0 

17 

23 

21 

21 

24 

12 

2 

18 

24 

25 

26 

26 

21 

0 

19 

3 

4 

10 

3 

13 

1 

21 

10 

14 

15 

17 

22 

5 

20 

14 

19 

20 

18 

25 

Time  Load  Rating 

Mental  Effort  Load  Rating 

Psychological  Stress  Load  Rating 


Table  22  SWAT  WORKLOAD  RATINGS,  Time  Load  Emphasis  Group 


LEVEL 

(T,M,P) 

WORKLOAD 

(1,1,1) 

0.0 

(1,1,2) 

14.2 

(1,1,3) 

20.1 

(1,2,1) 

15.7 

(1,2,2) 

29.9 

(1,2,3) 

35.7 

(1,3,1) 

26.2 

(1,3,2) 

40.4 

(1,3,3) 

46.3 

(2,1,1) 

30.0 

(2,1,2) 

44.2 

(2,1,3) 

50.1 

(2,2,1) 

45.7 

(2,2,2) 

59.9 

(2,2,3) 

65.7 

(2,3,1) 

56.2 

(2,3,2) 

70.4 

(2,3,3) 

76.3 

(3,1,1) 

53.7 

(3,1,2) 

67.9 

(3,1,3) 

73.8 

(3,2,1) 

69.4 

(3,2,2) 

83.6 

(3,2,3) 

89.5 

(3,3,1) 

79.9 

(3,3,2) 

94.1 

(3,3,3) 

100.0 

(T,M,P):  T  -  Time  Load  Rating 

M  -  Mental  Effort  Load  Rating 
P  -  Psychological  Stress  Load  Rating 


Table  24  SWAT  WORKLOAD  RATINGS,  Psychological  Stress  Load 
Emphasis  Group 


LEVEL 

(T,M,P) 

WORKLOAD 

(1,1,1) 

0.0 

(1,1,2) 

28.8 

(1,1,3) 

48.5 

(1,2,1) 

13.0 

(1,2,2) 

41.8 

(1,2,3) 

61.5 

(1,3,1) 

23.9 

(1,3,2) 

52.6 

(1,3,3) 

72.4 

(2,1,1) 

10.4 

(2,1,2) 

39.2 

(2,1,3) 

59.0 

(2,2,1) 

23.5 

(2,2,2) 

52.2 

(2,2,3) 

72.0 

(2,3,1) 

34.3 

(2,3,2) 

63.1 

(2,3,3) 

82.8 

(3,1,1) 

27.6 

(3,1,2) 

56.4 

(3,1,3) 

76.1 

(3,2,1) 

40.6 

(3,2,2) 

69.4 

(3,2,3) 

89.2 

(3,3,1) 

51.5 

(3,3,2) 

80.2 

(3,3,3) 

100.0 

(T,M, P) :  T 
M 
P 


Time  Load  Rating 

Mental  Effort  Load  Rating 

Psychological  Stress  Load  Rating 
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Table  25  SWAT  WORKLOAD  RATINGS,  Composite  Group  Solution 
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23.7 
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37.5 

(1,2,1) 

16.2 

(1,2,2) 

39.9 

(1,2,3) 

53.7 

(1,3,1) 

33.3 

(1,3,2) 

57.0 

(1,3,3) 

70.8 

(2,1,1) 

9.8 

(2,1,2) 

33.5 

(2,1,3) 

47.3 

(2,2,1) 

26.1 

(2,2,2) 

49.8 

(2,2,3) 

63.5 

(2,3,1) 

43.2 

(2,3,2) 

66.9 

(2,3,3) 

80.6 

(3,1,1) 

29.2 

(3,1,2) 

52.9 

(3,1,3) 

66.7 

(3,2,1) 

45.4 

(3,2,2) 

69.1 

(3,2,3) 

82.9 

(3,3,1) 

62.5 

(3,3,2) 

86.2 

(3,3,3) 

100.0 

APPENDIX  C 


WORKLOAD  SURVEY 


This  appendix  includes  the  workload  prediction  survey 
instructions  and  cockpit  equipment  description  for  the 
RAM/ACE  experiment  and  its  replication.  This  appendix  was 
not  created  by  the  author,  but  was  modified  slightly  and 
used  in  the  RAM/ ACE  experiment  replication.  It  is  included 
here  to  illustrate  the  degree  of  detail  available  in  the 
experiment  and  to  provide  a  reference  since  this 
information  is  not  otherwise  readily  available. 


PILOT  WORKLOAD  PREDICTIONS 


Instructions  and  Description 


LTC  Ton  Schuppe 

Department  of  Industrial  and  Systems  Engineering 
The  Ohio  state  University 
1971  Neil  Avenue 
Columbus*  Ohio  43210 


Introduction 


This  exercise  is  being  conducted  to  determine  a 
pilot's  ability  to  predict  his  workload  in  a  proposed 
aircraft  system.  You  have  been  asked  to  participate  in 
this  study  so  that  we  can  use  your  unique  experience  and 
insight  in  the  air-to-ground  combat  environment.  You  will 
be  provided  a  detailed  description  of  an  aircraft,  its 
subsystems,  and  a  mission  scenario.  Using  this  information 
and  your  own  flying  experience,  you  will  be  asked  to 
predict  pilot  workload.  The  information  you  provide  will 
form  an  essential  part  of  an  Air  Force-sponsored  research 
effort.  The  goal  of  this  effort  is  to  develop  a  technique 
to  accurately  estimate  pilot  workload  in  a  new  aircraft 
system  prior  to  the  development  of  hardware.  This  will 
provide  two  distinct  benefits:  more  affective  systems  will 
be  built  and  development  costs  will  be  reduced. 

The  aircraft  considered  in  this  exercise  is  an 
advanced  version  of  the  F-15  C/0.  It  will  have  several 
enhancements  designed  to  improve  its  performance  as  an  air- 
to-ground  weapon  delivery  system.  This  configuration  will 
be  designated  as  an  Advanced  Baseline  (ABL) .  After  reading 
a  detailed  description,  you  will  be  asked  to  provide  three 
sets  of  workload  estimates:  one  for  the  ABL  and  two  other 
sets  for  more  enhanced  versions.  The  first  of  these  two 
enhanced  versions,  using  an  automatic  Terrain  Following/ 
Terrain  Avoidance  (TF/TA)  subsystem,  will  be  described  and 


you  will  again  estimate  pilot  workload.  Your  final 
workload  estimates  will  be  for  the  second  enhanced  version, 
using  an  automatic  Terrain/Threat  Avoidance  (TTA)  subsystem 
added  to  the  ABL.  Consider  each  of  these  two  enhancements 
to  be  added  separately  and  independently  to  the  ABL. 

This  document  contains  a  combination  of  descriptive 
material  and  directions  to  lead  you  through  this  exercise. 
It  should  take  approximately  90  minutes  to  complete  this 
exercise.  If  possible,  try  to  do  the  entire  exercise  in 
one  sitting.  This  should  minimize  your  time  commitment  and 
also  increase  the  consistency  of  your  answers.  Everything 
you  are  being  asked  to  do  should  be  explained  in  detail. 
However,  if  you  have  any  questions,  please  contact  me  at 
any  time.  (LTC  Tom  Schuppe,  513-879-0616  or  614-292-4567) 

Workload 

Pilot  workload  is  an  important  consideration  in  the 
design  of  any  new  aircraft  system.  Several  methods  have 
been  developed  to  measure  pilot  workload.  One  of  these  is 
known  as  SWAT  (Subjective  Workload  Assessment  Technique). 
SWAT  relies  on  a  pilot's  ability  to  subjectively  assess  the 
workload  he  experienced  in  a  given  situation  after  it  has 
occurred.  It  has  been  widely  tested,  validated,  and 
accepted  as  a  meaningful  measure  of  pilot  workload.  In  the 
past  it  has  been  used  exclusively  with  a  pilot  in  a  flight 
simulatoz  or  aircraft  to  measure  his  perceived  workload. 


In  this  exercise,  we  are  now  attempting  to  use  it  to 
predict  workload  rather  than  measure  it. 

SWAT  assumes  there  are  three  independent  dimensions  to 
workload:  Time  Load,  Mental  Effort  Load,  and  Psychological 
Stress  Load.  Each  of  these  dimensions  may  assume  one  of 
three  levels:  low,  medium,  or  high.  These  ratings  are 
entirely  subjective  and  reflect  each  rater's  judgment 
concerning  pilot  workload.  A  complete  description  of  the 
ratings  in  each  dimension  is  found  on  the  next  page. 

You  will  be  asked  to  provide  ratings  in  each  dimension 
for  various  mission  and  equipment,  combinations.  One  word 
of  caution,  there  are  no  "right"  answers.  Each  pilot 
accomplishing  this  task  may  rate  workload  differently. 

Final  workload  predictions  are  based  on  a  combination  of 
these  ratings  and  your  previous  ranking  of  27  cards 
containing  workload  descriptions.  Therefore,  it  is  not 
important  how  your  answers  might  compare  with  someone 
else's  answers.  However,  it  is  important  that  you  are 
consistent  in  rating  workload  and  ranking  the  27  card 
descriptions  of  workload. 

If  you  normally  fly  a  single-seat  aircraft,  your 
workload  estimates  should  be  made  on  that  basis.  Likewise, 
if  you  fly  an  aircraft  with  a  second  crewmember,  your 
workload  estimates  should  reflect  only  your  own  workload  as 
you  now  accomplish  it.  This  may  require  you  to  ignore  some 
tasks  described  as  pilot  tasks  since  your  second  crewmember 


normally  accomplishes  these. 


Mission  Description 

The  mission  used  in  all  of  these  evaluations  will  be  a 
battlefield  interdiction  (BI)  mission.  We  are  mainly 
concerned  about  the  Penetration,  Target  Acquisition,  and 
Weapon  Delivery  segments,  as  shown  in  Figure  11.  The 
Penetration  segment  is  flown  at  200  feet  AGL,  at  600  knots 
and  lasts  approximately  5  minutes.  It  includes  penetration 
of  the  FEBA  and  its  associated  threats.  The  next  segment 
is  Target  Acquisition  and  consists  of  a  4  G  pullup  so  that 
sensors  may  locate  and  designate  the  target.  This  segment 
is  Weapon  Delivery  and  involves  all  pilot  actions  required 
to  obtain  an  accurate  weapon  release,  consider  this 
segment  to  last  30  seconds.  Once  again,  hostile  threats 
must  be  considered  in  this  segment. 

Assume  that  the  entire  mission  is  conducted  during  the 
day  and  the  weather  is  3000  feet  overcast  with  3  miles 
visibility.  The  terrain  for  your  mission  is  gently  rolling 
with  maximum  hilltops  500  feet  above  your  nominal  flight 
path.  Your  target  is  a  tank  column  on  a  road  and  the 
weapon  you  will  use  is  an  HR  Maverick.  You  have  recent 
information  on  your  target's  position  and  are  certain  of 
its  location  within  one-half  mile.  Also  assume  that  your 


aircraft  is  fully  operational  at  the  start  of  the 
penetration  segment  and  that  your  fuel  is  adequate  to 
complete  the  mission. 
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SWAT  RATING  VALUE  DESCRIPTIONS 

Rating  values  and  descriptions  of  each  are  provided 
below  for  the  three  workload  factors  we  are  interested  in 
for  this  study.  Please  refer  to  these  descriptions 
whenever  necessary  during  any  enhancement  rating. 

TIME  LOAD  (TL) 

VALUE  DESCRIPTION 

1.  Often  have  spare  time.  Interruptions  or 
overlap  among  activities  occur  infrequently 
or  not  at  all. 

2.  Occasionally  have  spare  time.  Interruptions 
or  overlap  among  activities  are  very 
frequent. 

3.  Almost  never  have  spare  time,  interruptions 
or  overlap  among  activities  are  very 
frequent,  or  occur  all  the  time. 

MENTAL  EFFORT  LOAD  (ML) 

VALUE  DESCRIPTION 

1.  Very  little  conscious  mental  effort  or 
concentration  required.  Activity  is  almost 
automatic,  requiring  little  or  no 
attention. 

2.  Moderate  conscious  mental  effort  or 
concentration  required.  Complexity  of 
activity  is  moderately  high  due  to 
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uncertainty,  unpredictability,  or 
unfamiliarity.  Considerable  attention 
required. 

3.  Extensive  mental  effort  and  concentration 

are  necessary.  Very  complex  activity 
requiring  total  attention. 

PSYCHOLOGICAL  STRESS  LOAD  (SL) 

VALUE  DESCRIPTION 

1.  Little  confusion,  risk,  frustration,  or 
anxiety  exists  and  can  be  easily 
accommodated. 

2.  Moderate  stress  due  to  confusion, 
frustration,  or  anxiety  noticeably  adds  to 
workload.  Significant  compensation  is 
required  to  maintain  adequate  performance. 

3.  High  to  very  intense  stress  due  to 
confusion,  frustration,  or  anxiety.  High 
to  extreme  determination  and  self-control 
required. 
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ABL  Instructions 

A  detailed  description  of  the  F-15  Advanced  Baseline 
(ABL)  is  presented  in  the  next  section.  Read  this,  keeping 
in  mind  the  mission  scenario  just  presented,  and  then  make 
your  best  estimate  of  pilot  workload.  Estimates  will  be 
required  for  each  segment  (Penetration,  Target  Acquisition, 
and  Weapon  Delivery)  in  each  dimension  (Time  Load,  Mental 
Effort  Load,  and  Psychological  Stress  Load) . 
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ADVANCED  BASELINE 

To  bring  the  RAM/ACE  baseline  F-15C/D  up  to  a 
configuration  consistent  with  its  expected  capabilities  for 
the  study  time  period,  an  advanced  baseline  (ABL) 
fighter/attack  system  was  created.  The  ABL  utilizes  the 
same  basic  airframe  and  engine  capabilities  demonstrated  by 
the  F-15C/D  with  the  exception  of  its  avionics  suite  which 
will  include  the  following: 

a)  A  "glass"  cockpit  to  include  multipurpose 
displays,  a  Head-Up  Display  (HUD),  and  an  up-front  control 
panel. 

b)  A  more  sophisticated  radar  system  which  includes 
Synthetic  Aperture  Radar  (SAR) . 

c)  Manual  Terrain  Following  (TF) . 

d)  Carriage  capability  for  the  Imaging  Infrared  (IIR) 
Maverick  and  WASP  anti-armor  munition. 

All  of  these  capabilities  being  added  do  not  affect 
the  basic  air-to-air  capabilities  of  the  existing  F-15C/D. 
However,  since  the  thrust  of  RAM/ ACE  is  air-to-ground 
tactical  attack,  the  air-to-air  capabilities  will  not  be 
exercised  during  these  simulations. 

The  system  enhancements  derived  during  this  study  will 
be  added  to  the  ABL  configuration  and  evaluated  in  this 
simulation. 


172 

Aircraft  Parameters 

The  Advanced  Baseline  (ABL)  is  based  on  the  F-15D 
airframe  with  many  avionics  and  crew  station  changes.  The 
aircraft  is  powered  by  two  advanced  turbofan  engines,  has  a 
high  mounted  sweptback  wing  and  two  vertical  stabilizers. 
The  airframe  parameters  are: 
o  Wingspan  -  43  feet 
o  Length  -  64  feet 

o  Height  -  top  of  vertical  tail  -  19  feet 
top  of  closed  canopy  -  12  feet 
o  Distance  between  main  landing  gear  -  9  feet 
o  Takeoff  gross  weight  -  full  internal  fuel, 
armament :  70,000  pounds 

Avionics/Sensor  Description 

The  Advanced  Baseline  aircraft  is  configured  with  an 
avionics  system  which  includes  a  large  memory,  high  speed 
digital  computer,  redundant  multiplexed  digital  data  buses, 
multifunction  programmable  displays  and  display  processors, 
a  wide  field-of-view  HUD  and  an  Up  Front  Control  (UFC) 
Panel.  The  onboard  sensors  include  a  high  resolution 
Forward  Looking  Infrared  (FLIR)  sensor.  The  operation  of 
the  multifunction  displays  and  up  front  control  are 
described  later. 

The  following  list  provides  some  of  the  important 
features  of  the  ABL  avionics  systems. 
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o  Fire  Control  System  -  Consists  of  the  radar  system, 
the  FLIR,  the  lead  computing  gyro  and  the  armament  control 
system.  This  provides  an  effective  all-weather  air-to- 
surface  weapon  delivery. 

°  Threat  Warning  System  -  Provides  full  spherical 
coverage  to  warn  of  air-to-air  and  surface-to-surface 
threats . 

o  Navigation  System  -  Consists  of  very  accurate 
Inertial  Navigation  System  (INS),  with  accuracies 
approaching  .5  NM  per  hour,  and  a  Global  Positioning 
System.  The  two  systems  provide  virtually  error  free 
navigation. 

o  Communication,  RF  Navigation  and  Identification  - 
Provides  for  voice  and  data  communications  and 
identification  of  self  and  other  aircraft,  as  well  as 
compatibility  with  existing  navigation  facilities. 

o  Digital  Computation  -  Consists  of  a  large  memory, 
high  speed  digital  computer  and  associated  redundant 
digital  multiplexed  data  buses. 

o  Controls  and  Displays  -  Consists  of  multiple 
programmable  CRT  displays  in  both  front  and  rear  cockpits, 
multiple  display  generators,  a  wide  field  of  view  HUD 
(front  only/,  and  up  front  controls. 

o  Flight  Controls  -  Quadruple  redundant  fail  operate, 
fail  safe  flight  control  system. 
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ADVANCED  BASELINE  RADAR 

The  resolution  and  quality  of  imagery  the  operator 
views  is  primarily  determined  by  the  antenna  beamwidth;  the 
narrower  the  beamwidth  the  better  the  resolution  and  image 
quality.  Two  factors  determine  the  beamwidth,  (1)  the  size 
of  the  radar  antenna  and  (2)  the  operating  frequency.  The 
larger  the  antenna  and/or  the  higher  the  frequency,  the 
narrower  the  beamwidth,  and  hence  the  better  the 
resolution.  Host  contemporary  fighter  and  attack  aircraft 
do  not  have  space  available  for  large  antennas,  and 
operating  frequencies  are  determined  by  other  system 
performance  requirements  such  as  missile  compatibility,  in 
weather  operation,  etc. 

If  the  radar  antenna  were  directed  to  stop  scanning 
and  look  at  a  fixed  point  on  the  ground,  and  the  target 
return  data  sampled  and  processed  as  the  aircraft  moved 
over  the  ground  (Figure  12)  it  would  in  effect 
synthetically  generate  an  antenna  as  big  as  the  distance 
the  aircraft  traveled  during  the  sampling  period.  This 
effect  produces  a  very  narrow  beamwidth  and  hence  very  high 
resolution  and  image  quality.  The  high  resolution  map 
mode,  referred  to  as  synthetic  aperture  radar,  produces 
near  optical  quality  imagery.  Because  of  the  generation 
techniques  employed,  i.e.,  aircraft  motion,  2  to  10  seconds 
(frame  time)  are  generally  required  to  form  the  map  images 
and  squint  angles  off  of  ground  track  are  required. 
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•  AZIMUTH  RESOLUTION  DETERMINED  IT  •  AZIMUTH  RESOLUTION  DETERMINED  SV 

SYNTHETIC  APERTURE  LENGTH  REAL  ANTENNA  LENGTH 

•  SYNTHETIC  APERTURE  LENGTH  ACHIEVED  •  LONGER  ANTENNA  LENGTH  PROVIDES 

•Y  COHERENT  DIGITAL  PROCESSING  SETTER  AZIMUTH  RESOLUTION  ' 


Figure  12  Synthetic  Aperture  Radar 
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Since  operation  of  this  mode  relies  on  doppler  effect 
processing,  the  mode  is  not  operable  for  targets  along  the 
aircraft  ground  track  or  so  near  to  it  that  the  frequency 
shift  is  too  small  for  processing.  In  the  ABL  this 
unusable  portion  is  approximately  +/-  10°  from  ground  track 
as  shown  in  Figure  13. 

Forward  Looking  Infrared  Sensor  (FLIR)  -  The  FLIR  used  on 
the  advanced  baseline  aircraft  is  a  pod  mounted  sensor  with 
two  fields  of  view,  12°  x  12°  wide  field  of  view  and  3°  x 
3°  narrow  field  of  view.  It  has  a  field  of  regard  of 
+30°,  -150°  elevation  and  +/“540°  roll.  The  FLIR  can  be 
cued  to  a  line  of  sight  as  designated  by  the  radar  or 
navigation  system,  and  has  an  inertial  tracking  mode. 

COCKPIT  LAYOUT 

The  forward  and  aft  cockpits  of  the  ABL  have  been 
modified  to  accommodate  added  systems  and  capabilities. 

The  forward  crew  station  incorporates  the  addition  of  three 
Cathode  Ray  Tubes  (CRT)  that  are  Multipurpose  Displays 
(MPD's)  with  20  pushbuttons  around  the  periphery.  The  left 
and  right  MPD's  are  6"  monochromatics  and  the  center  is  a 
5"  monochromatic.  Any  display  can  be  called  up  on  the  left 
and  right  MPD's  and  used  for  systems  management.  The  lower 
center  MPD  can  be  used  to  display  the  Electronic  Horizontal 
Situation  Indicator  (EHSI)  and  the  Head  Down  Display  (HDD) . 
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The  forward  crew  station  configuration  is  shown  in  Figure 
14. 

While  nost  of  the  F-15C/D  sensor  console  control 
panels  are  retained,  the  majority  of  these  panels  are 
backup  controls.  This  is  accomplished  by  extending  the 
capabilities  of  the  hands-on  control  concept  of  performing 
aoding  and  subsystem  control  utilizing  displayed  options  on 
the  CRT  display  and  selecting  these  options  via  display 
mounted  pushbuttons,  or  by  using  a  cursor  controlled  from 
the  throttle  grips.  Cursor  control  is  assigned  to  a 
particular  display  using  an  added  switch  on  the  flight 


Figure  14  RAM/ACE  Advanced  Baseline  Forward  Crew  Station 
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control  stick  grip.  Cursor  slewing  tnd  option  selection  is 
done  with  the  Target  Designator  Controller  (TDC)  on  the 
throttle. 

The  forward  and  aft  crew  stations  also  incorporate  an 
Up-Front  Control  (UFC)  panel.  The  UFC  features  a  keyboard 
for  data  entries  associated  with  communications, 
navigation,  and  identification  functions  on  the  aircraft. 
The  onboard  radios  and  Identification,  Friend  or  Foe  (IFF) 
functions  are  also  integral  to  the  UFC, 

The  aft  crew  station  consists  of  two  6"  monochromatic 
MPD's  in  the  center  and  two  5"  monochromatic  MPD's  on 
either  end.  Display  control  in  the  aft  crew  station  is 
totally  independent  from  that  of  the  front  cockpit  and  vice 
versa.  Two  stationary  hand  controllers  located  on  left  and 
right  side  consoles  give  the  weapon  system  officer  (WSO) 
control  of  all  aft  seat  functions. 

Primary  subsystem  control  is  via  control  menus  or 
lists,  presented  on  the  MPD's.  The  WSO  can  either  use 
associated  pushbuttons  or  displayed  cursors,  controlled 
from  the  hand  controls,  to  select  the  operating  modes  for 
each  subsystem.  He  also  has  full  control  of  the  setup  of 
the  four  display  units.  He  can  quickly  change  the  display 
of  any  subsystem  from  one  MPD  to  another  or  change  which 
subsystems  are  being  presented.  The  aft  crew  station  is 
illustrated  in  Figure  15. 


Figure  15  RAM/ACE  Advanced  Baseline  Aft  Crew  Station 
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CONTROLS  OPERATION 

Forward  Cockpit  Stick  and  Throttle  Functions: 

In  order  to  allow  rapid  weapon  system  control  in  time 
critical  situations ,  the  RAM/ACE  ABL  crew  stations  were 
designed  with  many  weapon  system  control  functions  on  the 
stick  and  throttle  in  the  forward  crew  station  and  on  the 
two  stationary  hand  controllers  in  the  aft  crew  station. 
These  functions  are  outlined  below  and  in  Figures  16 
through  19. 

Stick  Functions; 

Pickle  Button  -  When  Maverick  is  the  selected  weapon, 
each  depression  releases  one  AGM-65D  Imaging  Infrared  (HR) 
Maverick.  When  WASP  is  the  selected  weapon,  holding  the 
pickle  button  depressed  will  ripple  off  WASP  missiles  until 
weapon  count  goes  to  zero. 

TDC  Assignment  Switch  -  The  TDC  assignment  switch  is  a 
4-position  momentary  action,  return  to  center  switch.  The 
forward  position  assigns  TDC  functions  to  a  HUD  cursor  for 
the  purpose  of  visual  designations  or  designation  updates. 
The  left  position  assigns  TDC  functions  to  the  left  MPD  and 
its  cursor  for  the  purpose  of  sensor  control  and 
designation.  The  right  position  assigns  TDC  functions  to 
the  right  MPD  and  its  cursor.  The  aft  position  assigns  TDC 
functions  to  the  lower  center  MPD  and  its  cursor. 

Trim  Switch  -  Activates  aileron  and  elevator  trim 


functions. 
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Maverick/FLIR  Switch  -  This  is  a  three  position 
switch: 

Forward  -  Subsequent  depressions  alternates  FUR 
Field-of-View  (FOV)  between  wide  and  narrow. 

Down  -  Alternates  between  the  FLIR  and  Maverick 
display  on  the  same  MPD. 

Aft  -  Steps  between  Maverick  Weapon  stations  in 
lieu  of  using  the  armament  control  display.  When  the 
boresight  correlator  option  is  selected  on  the  FLIR 
display,  activation  of  this  switch  steps  the  priority 
target  between  those  selected  by  the  FLIR. 

Undesiqnate  Switch  -  Activation  cancels  a  previously 
accomplished  designation. 

Paddle  Switch  -  Disengages  all  autopilot  and  terrain 
following  functions. 

Throttle  Functions: 

UHF  Comm  Switch  -  To  transmit  on  UHF1  depress  and 
hold  this  switch  aft. 

Speed  Brake  Switch  -  Depressing  and  holding  this 
switch  aft  extends  the  speed  brake.  Depressing  and  holding 
this  switch  forward  retracts  the  speed  brake. 

Voice  Recognition  Command  (VRC)  switch  - 
Depressing  and  holding  this  switch  forward  keys  the  voice 
recognition  system  enhancement. 

Coordinate  Transmit  Switch  -  During  the  multi¬ 


ship  operations  enhancement,  activation  of  this  switch 
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Figure  16  RAN/ACE  Forward  Craw  Stick  Functions 

transaits  target  coordinates  to  the  wingman. 

Pop-Up  Command  Switch  -  Moving  the  switch  up 
coaaands  the  pop-up  mode  nOL  cubing  and  cancels  terrain 
following  HUD  coaaands.  Pushing  the  switch  down  cancels 
the  pop-up  cueing. 

Target  Designator  Controller  (TPC)  -  In  the  non- 


depressed  {no  action)  position,  the  TDC  slews  display 
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Figure  17  RAH/ACE  Forward  Crew  station  Throttle  Functions 

cursors ,  sensors  or  the  Maverick  head  depending  on  the  TDC 
assignment.  When  depressed  and  released,  the  TDC  commands 
designation  Maverick  lock-on,  or  KPD  display  option. 

Waypoint  Step  Button  -  Each  depression  increments 
to  the  next  prestored  waypoint  data  for  steering 
information. 

Radar  Elevation  control  -  A  potentiometer  that 
moves  the  radar  scan  in  elevation  up  or  down. 

Chaff  Dispensing  Switch  -  Each  activation  of  the 
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switch  deploys  one  chaff  bundle. 

Aft  Crew  Station  Controls 

Although  the  switches  used  on  the  hand  controllers, 
Figures  18  and  19,  in  the  aft  crew  station  are  different 
from  the  front  seat,  the  functions  are  identical. 

The  left  and  right  hand  controller  functions  are 
identical  and  mirror  images  of  one  another.  The  exception 
to  the  rule  are  the  TDC  assignment  switches.  Because  there 
are  four  MPD's  in  the  aft  crew  station,  the  TDC  assignment 
switch  on  the  left  hand  controller  assigns  TDC  fucntions 
between  the  two  left  MPD's  and  the  TDC  assignment  switch  on 
the  right  hand  controller  assigns  TDC  functions  between  the 
two  right  MPD's. 

The  WSQ  can  release  weapons  from  the  aft  seat  using 
the  weapon  release  button  on  the  stick.  The  remainder  of 
the  stick  switches  in  the  aft  crew  station  are  non¬ 
functional.  The  UHF  coma  switch  on  the  aft  crew  station 
throttle  is  also  operational.  Again,  this  is  the  only  aft 
cockpit  throttle  switch  used. 

Up-Front  Control  (UFC)  operation 

The  UFC  used  in  RAM/ACE  is  illustrated  in  Figure  20 
along  with  the  top  level  functions.  Four  systems  are 
controlled  via  the  UFC.  They  are  Identification  Friend  or 
Foe  (IFF),  waypoint  data  entry  and  control,  the  three 
terrain  following  modes,  and  UHF  radios  (both  numeric  and 


Figure  18  RAH/ACE  Aft  Crew  Station  Left  Hand  Controller 
Functions 

preset  channels) .  The  center  numeric  keyboard  is 
operational  along  with: 

CLR  -  clear  scratchpad  data 
MENU  -  Return  to  top  level  UFC  format  from 
sublevel  format 

SHF  «  Allows  use  of  the  N,  S,  E,  W  and  decimal 
point  functions  on  the  UFC  keyboard 
N,  S  **  Latitude  coordinate  prefixes 
E,  w  -  Longitude  coordinate  prefixes 
.  -  Decimal  point  for  freguency  entries 

One  UHF  radio  is  available  in  the  ABL:  UHF  1  (left 
side).  Ten  (10)  preset  radio  channels  (1*10)  are  available 
for  the  radio.  To  change  a  UHF  frequency,  rotate  the  radio 
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Figure  19  SAM/ACE  Aft  Crew  station  Right  Hand  Controller 
Functions 


channel  select  knob  to  the  desired  channel.  Enter  the 
frequency  of  choice  in  the  scratchpad  (225.000  to  399.975). 
Once  the  frequency  is  correct *  depress  “ENTER".  That 
frequency  is  nos  entered  into  that  preset  channel. 

When  the  IFF  pushbutton  is  selected  on  the  top  level 
UFC  format,  the  IFF  sublevel  appears  as  shown  in  Figure 
21.  The  IFF  sodes  available  in  this  study  are  nodes  M3N, 
"4N,  and  "C".  Node  3  is  automatically  enabled  in  the 
scratchpad  for  data  entry.  To  change  the  node  3  squawk, 
siaply  type  in  the  four  number  code  and  depress  enter.  The 
new  code  will  appear  in  the  mode  3  row.  Node  4  has  two 
allowable  codes:  4A  and  4B.  Each  time  the  associated 
option  select  switch  is  depressed,  the  mode  4  code  scrolls 
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Figure  20  RAN/ ACE  Up-Front  control 

from: 

4A  :4A  4B  :4B  4A 

(4A-OFF)  (4A-ON)  (4B-OFF)  (4B-ON)  (4A-0FF) 

Node  c  is  either  on  (:)  or  off.  The  option  select 
switch  works  as  an  alternate  action  button.  To  return  to 
the  top  level  display,  sittply  depress  the  "MENU"  button  and 
the  selected  IFF  aodes  are  displayed. 

Upon  selecting  the  "WVPT"  option  select  button  or  the 
UFC  button  on  the  EK&X  WVPT  sublevel,  the  UFC  waypoint 
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Buttons  Next  to  IFF  Options 
Used  to  Turn  On  That  Option 
(:)  •  Indicates  “On" 


Mode  4  Sequence: 
4A-:4A4B-:4B4A4tc 


Keyboard  Entries  Used 
Only  tor  Mode  3 


Figure  21  RAM/ ACE  UFC  IFF  Sublevel 

sublevel  display  appears  as  shown  in  Figure  22.  This 
format  shove  the  current  latitude  and  longitude  for  the 
selected  waypoint  coordinates  can  be  entered  by  depressing 
the  "POSH"  option  select  button  {a  colon  appears)  which 
enables  the  scratchpad  for  data  entry.  For  a  latitude  the 
coordinates  oust  be  prefaced  by  a  SHF  “N*  or  SHF  "S*  and 
for  a  longitude  the  coordinates  must  be  prefaced  by  a  SHF 
**EM  or  SHF  *W".  The  coordinate  entry  format  east  follow 
the  illustrated  example  of  Figure  23. 
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Two  number  second 

entry  (e.g.,  22  ) —  . . — . — . — ■ 

Figure  23  RAM/ ACE  Waypoint  Data  Entry  Format 

Once  the  pilot  is  satisfied  with  the  scratchpad  entry, 
he  depresses  "ENTER1*  and  the  new  position  is  entered  and 
f • sprayed.  The  pilot  can  enter  the  elevation  for  each 
waypoint  in  a  similar  manner.  Valid  entries  are  from  0 
feet  to  999  feat. 

The  pilot  can  also  sequence  through  each  waypoint 
using  the  up  and/or  down  arrows  on  the  UFC  waypoint 
sublevel  display  or  EHSI  sublevel  or  top  level  displays. 
However,  changing  waypoint  numbers  on  either  the  EHSI  or 
UFC  sublevel  displays  will  not  change  the  waypoint  number 
selected  on  either  top  level  format.  To  return  to  the  UFC 
top  level  format,  the  pilot  selects  "MENU". 

If  the  option  select  button  for  one  of  the  terrain 
following  modes  is  depressed  when  in  the  UFC  top  level 
format,  the  terrain  following  sublevel  becomes  available  as 
shown  in  Figure  24. 
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—  Auto'Manual  it  Applicable  to  TF/TA  and  TTA 

—  TF  Is  Always  Manual 

—  CLNC  is  tha  Only  Kayboard  Entry 


Figure  24  RAM/ ACE  UFC-TF/TA  Sublevel 

The  options  are: 

CLNC  -  Allows  the  operator  to  select  the  terrain 
clearance  plane  for  TF  operations. 

AUTO  -  Automatic  terrain  following  mode  applicable 
to  TF/TA  or  TTA  and  mutually  exclusive  with  MANUAL  mode. 

MAN  -  Manual  terrain  following  mode  available  with 
TF/TA,  TTA,  and  TF.  Mutually  exclusive  with  Automatic 
mode. 

TF  -  Terrain  following  mode.  The  only  terrain 
following  mode  available  on  the  ABL.  It  offers  elevation 
commands  only. 

TF/TA  -  Terrain  following  and  terrain  avoidance  mode. 


*  v  w  ^ .  .vwv  aaut « « v a 
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Both  pitch  and  roll  commands  for  terrain  avoidance  are 
provided.  This  option  will  only  be  operational  with  the 
TF/TA  enhancement. 

TTA  -  Terrain  and  threat  avoidance  mode.  Both 
pitch  and  roll  real-time  commands  for  the  best  path  between 
both  terrain  and  detected  threats  is  given.  This  option 
will  only  be  available  when  the  TTA  enhancement  is  run. 
NOTE:  TF,  TF/TA,  and  TTA  are  all  mutually  exclusive. 

To  return  to  the  top  level  UFC  format,  select  "MENU". 

ADVANCED  BASELINE  DISPLAYS 
Head-Up  Display  (HUD) : 

The  ABL  Head-Up  Display  is  a  Wide  Field-Of-View  (WFOV) 
diffractive  optics  KUD  that  yields  a  total  FOV  of  30°  in 
azimuth  and  approximately  20°  in  elevation.  An 
illustration  of  the  HUD  is  shown  in  Figure  25.  The  basic 
symbology  used  on  the  HUD  is  shown  in  Figure  26.  In  the 
following  paragraphs,  thu  number  refers  to  the  callouts  in 
Figure  26. 

1.  Heading  -  The  aircraft's  magnetic  heading  is 
indicated  by  the  moving  360°  headirg  scale.  The  actual 
aircraft  heading  is  directly  above  the  caret.  The  moving 
heading  scale  provides  trend  information  during  turns.  As 
the  aircraft  turns  right,  the  scale  moves  from  right  to 
left. 

2.  Air  Speed  -  calibrated  air  speed  from  the  air  data 
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Figure  25  RAM/ ACE  Wide  Field-of-View  HUD 


computer  is  provided  in  the  box  on  the  left  side  of  the 
HUD.  The  tops  of  the  airspeed  and  altitude  boxes  are 
positioned  at  the  aircraft  waterline  (4°  up  from  the 
optical  center  of  the  HUD) . 

3.  Altitude 

4.  Angle  of  Attack  -  True  angle  of  attack  in  degrees 
is  displayed  at  the  left  center  of  the  HUD. 

5.  Mach  Number  -  The  aircraft  Mach  number  is 
displayed  immediately  below  the  digital  angle  of  attack. 

6.  Aircraft  G‘s  -  Normal  acceleration  of  the  aircraft 
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Figure  26  RAM/ ACE  HUD  Symbology 


is  displayed  immediately  below  the  Mach  number. 

7.  Velocity  Vector  -  The  velocity  vector  provides  the 
pilot  with  an  outside  world  reference  with  regard  to  actual 
aircraft  flight  path.  The  velocity  vector  represents  the 
point  towards  which  the  aircraft  is  flying  (aircraft  flight 
path) .  The  position  of  the  velocity  vector  is  limited  to 
an  8°-radius  circle  centered  at  the  HUD  optical  center.  If 


\ 
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the  velocity  vector  reaches  this  limit  during  high  angle  of 
attack  flight  or  large  yaw  and/or  drift  angles,  then  it 
will  flash  rapidly  (2.5  times  per  second)  to  indicate  that 
it  does  not  accurately  indicate  flight  path. 

8.  Flight  Path  Ladder  -  The  vertical  flight  path 
angle  of  the  aircraft  is  indicated  by  the  position  of  the 
velocity  vector  on  the  flight  path/pitch  ladder.  The 
horizon  and  flight  path/pitch  angle  lines  represent  the 
horizon  and  each  5°  of  angle  between  the  horizon  line. 
Negative  pitch  lines  are  dashed  and  are  below  the  horizon 
line.  The  outer  segments  of  the  lines  point  toward  the 
horizon.  Each  line  is  numbered  to  aid  in  determining 
flight  path  angle  when  it  is  changing  rapidly.  The  pitch 
lines  are  angled  toward  the  horizon  at  an  angle  half  that 
of  the  flight  path  angle.  For  example,  in  a  48°  climb,  the 
pitch  lines  are  angled  24°  toward  the  horizon.  In  level 
flight,  the  pitch  lines  are  not  angled.  The  zenith  is 
indicated  by  a  circle  and  the  nadir  is  indicated  by  a 
circle  with  an  X  in  it.  Aircraft  pitch  angle  can  be 
determined  by  comparing  the  tops  of  the  altitude  and 
airspeed  boxes  (which  represent  the  aircraft  waterline) 
with  the  pitch  ladder  when  the  wings  are  level.  However, 


since  the  flight  path/pitch  ladder  normally  rotates  about 
the  velocity  vector,  determination  of  pitch  angle  may  be 
difficult  at  high  roll  angles. 


Because  of  the  large  amount  of  data  that  can  be 
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displayed  on  the  HUD  at  any  one  time,  there  are  two  levels 


of  clutter  reject  provided  for  pilot  selection  via  the 
clutter  reject  switch  located  at  the  bottom-left  of  the  HUD 
control  panel.  The  clutter  reject  levels  provide  the  pilot 
the  option  to  remove  symbology  from  the  HUD  display  that  he 
considers  unessential  for  mission  completion  or  to  reduce 
interference  light  output.  The  symbology  that  is  removed 
from  the  display  is: 

Clutter  Reject  No.  1  Clutter  Reject  No.  2 

Aircraft  Mach  Number  Clutter  Reject  No.  1 

Symbology 


Aircraft  G*s 

Peak  Positive  G*s  Boxes  Around  Altitude  and 

Airspeed 

Heading  Scale  Nav  Range 

9.  Waypoint  Course  Arrow  -  When  waypoint  steering  is 
selected  on  the  EHSI  and  a  waypoint  course  is  set,  the 
waypoint  course  arrow  appears  on  the  HUD  showing  a  course 
depiction  similar  to  that  on  the  EHSI. 

10.  NAV  Range  -  When  waypoint  steering  is  selected  on 
the  EHSI,  the  range  to  waypoint  is  displayed  on  the  HUD. 
Upon  designation  of  a  target,  this  waypoint  range  changes 
to  target  range  which  illustrates  distance  to  weapon 
release  point. 

11.  Command  Heading  Bug  -  When  waypoint  steering  is 
selected  on  the  EHSI,  great  circle  steering  is  supplied  via 
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the  command  heading  bug  on  the  heading  scale. 

12.  Vertical  Velocity  -  Vertical  velocity  in  feet  per 
minute  is  displayed  above  current  altitude  on  the  HUD. 

The  WFOV  HUD  used  on  the  ABL  is  also  video  raster 
capable.  As  a  result,  FLIR  imagery  can  be  displayed  on  the 
HUD  to  aid  navigation. 

After  a  target  designation  (as  a  result  of  using  any 
sensor  or  the  HUD) ,  has  been  performed,  the  HUD  display 
changes  as  shown  in  Figure  27.  The  target  designation 
diamond  should  overlay  the  target  (if  not,  it  can  be 
updated  visually  or  with  a  designation  sensor  or  weapon) . 
The  pilot  should  steer  to  put  the  velocity  vector  over  the 
azimuth  steering  line  (ASL)  to  null  out  any  azimuth  errors. 
Range  to  target,  time  to  weapon  release  and  delivery  mode 
are  also  displayed  on  the  designated  HUD. 

Multipurpose  Display  Logic 

The  left  and  right  6M  monochromatic  MPD's  in  the  front 
cockpit  can  be  used  to  display  any  of  the  ABL's  formats. 

The  center  5H  monochromatic  MPD  in  the  front  seat  can  only 
display  the  EHSI  and  the  HUD  if  selected  from  the  EHSI.  An 
illustration  of  the  display  logic  is  shown  in  Figure  28. 

In  the  aft  crew  station  all  four  displays  are  MPD's 
and  can  be  used  to  project  any  format  that  is  desired.  On 
the  MPD's  the  operator  can  call  up  any  display  format  via 
the  MENU.  The  MENU  option  is  always  the  center  button  on 
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Figure  27  RAM/ACE  Basic  HUD  Display  Designated 

the  lover  row  of  buttons.  The  MENU  format  displays  all 
available  system  format  options  for  pilot  selection  as 
shown  in  Figure  29. 

Upon  selecting  a  format  using  the  option  buttons ,  the 
MENU  format  is  replaced  with  the  selected  format.  It's 
important  to  note  that  a  given  format  cannot  be  displayed 
on  two  MPD's  simultaneously  in  the  same  crew  station. 
However,  since  the  front  and  rear  cockpit  display  logic  is 
independent,  the  same  format  can  be  displayed 
simultaneously  in  both  the  front  and  rear  crew  stations. 


°i«play  Logic 
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Figure  29  RAH/ ACE  Mem  ■  Format 

The  available  display  formats  will  now  be  discussed  along 
with  their  associated  control  logic. 

Head  Down  Display  (HDD) 

The  HDD  represents  a  futuristic  combination  of  the 
contemporary  E-scope  terrain  following  display,  as  in  the 
F-lll,  and  the  newer  PPX  formatted  terrain  avoidance 
display  as  used  in  the  F/A-18A,  The  HDD  is  depicted  in 
Figure  30.  The  display  can  only  be  selected  via  a  button 
option  on  the  EHSI  format  and  represents  a  5.6  mile  X  5.6 
mile  area. 
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Figure  30  RAK/&CE  Head-Down  Display  (HDD) 


The  HDD  depicts  ovnahips  position  at  the  lower  center 
of  ths  display  just  above  tha  aircraft's  currant  heading. 
The  aircraft's  computed  pstn  is  shown  by  a  dashed  line 
while  tha  great  circle  direct  path  between  waypoints  is 
shown  by  the  solid  line*  Terrain  contours  are  shown  at  500 
feet  in  increments  of  terrain  elevation  and  are  shown  on 
the  HDD  as  follows: 

Dashed  line  -  represents  terrain  contour  below  the 
aircraft's  present  altitude 

1st  solid  line  -  represents  terrain  contour  above  the 


aircraft's  present  altitude 


At  the  lower  left  of  the  HDD  is  an  attitude  format 
referenced  to  a  fixed  aircraft  symbol  in  the  center  of  the 
circle.  At  the  lower  right  of  the  display  is  a  relative 
altitude  reference  illustrating  the  aircraft’s  present 
altitude  with  respect  to  the  next  level  of  terrain  above 
and  below  the  aircraft.  To  return  to  the  EHSI  simply 
depress  the  "HSI"  option  on  the  lower  right.  Or,  if  return 
to  the  menu  display  is  desired,  the  "MENU"  option  should  be 
selected. 

Armament  Control  System  (ACS)  Display 

The  ACS  format  represents  an  aircraft  wing-form  with 
the  available  weapons  and  sensors  being  displayed  on  their 
respective  stations.  On  stations  1  and  9,  AIM-9 
Sidewinders  are  always  shown,  even  though  they  cannot  be 
accessed  or  fired  in  the  HAM/ ACE  simulation.  The  FLIR  pod 
is  always  loaded  on  station  4.  The  master  arm  status  is 
shown  at  the  center  of  the  format  as  either  *ARMM  or 
“SAFE*.  There  are  two  different  weapon  loads  used  on  the 
ABL  in  RAH/ ACE:  the  first  is  an  HR  Maverick  where 
missiles  are  on  stations  2,  3,  ?,  and  8,  as  shown  in  Figure 
31.  To  get  a  weapon  ready,  the  following  functions  must  be 
accomplished. 

a)  The  Maverick  weapon  option  (MAV)  must  be  selected 
on  the  upper  row,  far  left  button.  When  this  is 
accomplished,  the  simulated  ACS  will  automatically  select  a 
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Figure  31  RAM/ ACE  ACS  Format,  Maverick  Weapon  Load 

weapon  station  for  first  launch,  indicated  by  a  box  around 
the  weapon  acronym.  The  operator  can  change  that  station 
by  depressing  the  step  option  as  many  times  as  is 
necessary. 

b)  The  "MODE"  option  must  be  selected  so  that  the 
system  will  enable  the  automatic  firing  mode. 

c)  The  air-to-ground  master  mode  must  be  selected  on 
the  main  instrument  panel. 

d)  The  Master  Arm  must  be  placed  in  "ARM". 

e)  In  the  case  of  the  Maverick,  the  "cage"  button 
must  be  depressed  on  the  Maverick  display  to  uncage  the 


weapon.  For  the  Boresight  correlator  enhancement,  the 
weapon  uncaging  is  automatically  accomplished  without  pilot 
action. 

After  ail  of  these  steps  have  been  accomplished,  the  X 
through  the  weapon  type  will  be  removed  from  the  ACS 
display,  the  HUD,  and  the  Maverick  display,  and  the  weapon 
release  button  becomes  hot.  The  second  ACS  weapon  load 
used  in  RAM/ACE  is  2  WASP  pods  containing  8  missiles  each, 
as  shown  in  Figure  32.  The  WASP  pods  are  loaded  on 
stations  2  and  8.  To  enable  the  weapons  for  launch,  the 
same  steps  must  be  followed  as  for  Maverick  except  no 
weapon  uncaging  is  required. 

Threat  Display 

To  use  the  threat  display,  select  the  "THRTM  option  on 
the  menu  format.  The  threat  display  is  shown  in  Figure 
33.  It  is  a  range  azimuth  format  that  depicts  ownship 
position  in  the  center  with  15  and  30  NM  range  rings.  It 
is  assumed  that  accurate  azimuth  and  range  is  determinable 
for  detected  threats.  A  threat's  centroid  is  shown  on  the 
display  by  placing  one  of  the  symbols  shown  in  Figure  34  at 
the  appropriate  range  and  azimuth.  Symbols  are  placed  on 
the  display  for  all  threats  which  have  you  in  detection 
range.  In  addition  to  a  threat's  centroid,  a  lethal  radius 
that  decreases  with  decreasing  ownship  altitude  is 
depicted.  The  lethal  radius  can  be  made  to  disappear  by 
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Figure  32  RAH/ ACE  ACS  Format,  WASP  Weapon  Load 

flying  very  low.  This  is  a  simple  method  of  simulating  the 
effects  of  terrain  masking.  However,  masking  due  to 
specific  topographic  features  is  not  included.  There  are 
three  threat  states  depicted.  These  are: 

o  Active  (non-tracking) ;  1  second  status  change  tone 
and  the  threat  symbol  with  it  lethal  radius  is  displayed. 

o  Tracking  (radar  tracking  ownship) ,  ownship  is 
within  threat  detection  range  (not  displayed) ;  continuous 
tracking  tone. 

o  Launch;  ownship  is  within  lethal  range;  continuous 
launch  tone.  Threat  symbol  and  lethal  range  circle  flash 


SA<6  in 
Missile  Track 


Figure  33  RAN/ACE  Threat  Display 


indicate  missile  launch  (or  gunfire) . 

Forward  Looking  Infrared  (FLIR)  Display 

The  FLIR  display  can  be  selected  via  the  MENU  format. 
It  is  illustrated  in  Figure  34.  When  the  FLIR  is  on  and 
operating  correctly,  it  is  indicated  by  an  "OPR"  in  the 
upper  left  corner  of  the  display.  In  the  center  of  the 
display,  a  velocity  vector,  horizon  line  and  a  field  of 
view  reticle  is  depicted.  The  azimuth  and  elevation 
position  of  the  FLIR  head  are  shown  at  top  center  and  left 
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Figure  34  RAH/ACE  FLIR  Display 

center  of  the  display  respectively.  The  remaining  FLIR 
options  are  discussed  below: 

WIDE/NAR  -  This  is  an  alternate  action  button  that 
changes  the  FLIR  field-of-view  (FOV) .  The  FLIR  FOV 
switches  on  the  stick  in  the  front  seat  and  hand 
controllers  in  the  rear  seat  duplicate  this  function.  The 
FLIR's  FOVs  are  12°  (wide)  and  3°  (narrow) . 

NOTE:  The  wide  FOV  reticle  with  the  short  perpendicular 
tic  marks  indicates  the  area  of  narrow  FOV  if  selected. 

A/G  WPN  -  Selects  the  Maverick  display  if  available. 
PRI  STEP  -  Steps  priority  targets  for  the  Bores ight 


correlator  enhancement  only. 

NOTE:  The  FOV  reticle  automatically  returns  upon 
designation  to  aid  the  designation  refinement  process. 

CAGE  -  Cages  the  FLIR  head  to  0°  azimuth  and  -7°  in 
elevation. 


DCLTR  -  Removes  the  velocity  vector  and  horizon  line 
as  well  as  airspeed,  Mach  and  altitude. 

To  move  the  FLIR  head,  TDC  functions  must  be  assigned 
to  the  FLIR  display  by  using  the  TDC  assignment  switch  in 
either  the  front  or  rear  cockpits.  A  diamond  will  appear 
in  the  upper  right  corner  of  the  display  indicating  TDC 
control.  To  slew  the  FLIR  head,  apply  forces  to  the  TDC 
without  depressing  it.  To  designate  a  target,  depress  the 
TDC  to  the  action  position  and  then  release  it.  The  FLIR 
should  now  be  ground  stabilized  on  whatever  is  located 
under  the  center  of  the  FOV  reticle.  Slewing  can  also  be 


accomplished  while  the  TDC  is  held  in  the  action  position 
for  designation  refinement  and/or  updating. 


Radar  Display 

The  available  radar  modes  in  the  ABL  are  Real  Beam 
Ground  Map  (RBGM)  and  10  ft.  resolution  SAR  (EXP-l) .  The 
RBGM  air-to-ground  radar  format  used  in  RAM/ ACE  is  shown  in 
Figure  35.  It  is  a  +70°  Plan  Position  Indicator  (PPI) 
format  with  four  equal  range  arcs.  On  the  left  side  of  the 
display  is  the  radar  elevation  caret  (<)  and  optimum 


Figure  35  RAM/ ACE  Radar  Real  Bean  Ground  Map  Display 
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elevation  mark  (-) .  The  scale  represents  +30  in 
elevation.  All  of  the  functions  around  the  periphery  of 
the  radar  format  can  be  selected  via  the  buttons  and  using 
the  Hands  on  Throttle  and  Stick  (HOTAS)  cursor.  To  select 
an  option  using  the  HOTAS  cursor,  place  the  cursor  over  the 
option,  then  depress  and  release  the  TDC. 

The  display  options  on  the  radar  format  are  outlined 
below: 

OPR  -  Indicates  that  the  radar  is  on  and  operating. 

MAP  -  Indicates  that  the  current  operating  mode  is 

RBGM. 


120°  -  Current  azimuth  scan.  The  azimuth  scan  can  be 
changed  in  one  of  two  ways.  First,  the  button  below  the 
current  azimuth  scan  can  be  pressed,  each  time  scrolling  to 
a  new  number.  Or,  the  HOTAS  cursor  can  be  placed  down  on 
the  lower  left  of  the  format  and  all  of  the  azimuth  scan 
options  automatically  appear  for  selection  by  the  HOTAS 
cursor . 

DCLTR  -  Removes  the  velocity  vector  and  horizon  line. 

80  -  The  up  arrow  increments  the  range  scale  and  the 
down  arrow  decrements  the  range  scale.  The  available  range 
scales  are  10,  20,  40,  80,  and  160  nautical  miles. 

EXP  1  -  When  designated,  selects  the  10  ft.  resolution 
SAR  mode  centered  around  the  designated  point. 

-  When  undesignated,  selects  the  invideo  EXP1 
outline  that  defines  the  10  ft.  resolution  SAR  patch.  This 
outline  is  slewable  within  the  radar  video  and  once  the  TDC 
is  depressed  to  the  action  position  and  released,  the  10 
ft.  SAR  mode  (EXP-1)  is  entered  about  the  center  of  the 
EXP-1  cursor.  The  process  is  illustrated  in  Figure  36. 

EXP  2  -  Select  Very  Higt\  Resolution  (VHR)  SAR  imagery 
only  during  the  SAR  enhancements. 

To  designate  a  target  on  the  radar  display,  the 
following  procedure  must  be  used.  Use  the  TDC  to  position 
the  HOTAS  cursor  within  the  radar  video  presentation. 
Depress  and  hold  the  TDC.  A  radar  cursor  (in  video) 
replaces  the  HOTAS  cursor.  While  holding  in  the  TDC,  move 
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the  radar  cursor  over  the  desired  target  and  release  the 
TDC.  The  radar  cursor  is  replaced  by  a  stabilized  invideo 
cue  showing  the  location  of  the  designated  target.  The 
HOTAS  cursor  returns  to  the  display.  If  any  other  source 
(FLIR,  navigation  system,  etc)  is  used  to  designate  a 
target,  a  stabilized  cue  will  also  appear  in  the  radar 
video.  This  procedure  is  used  in  both  MAP  and  EXP1  radar 
modes,  and  is  shown  in  Figure  37.  If,  as  you  get  closer  to 
the  target,  a  refinement  of  the  previous  designation  is 
desired,  the  same  procedure  as  the  original  designation 
should  be  followed. 

Electronic  Horizontal  Situation  Indicator  (EHSI) 

The  EHSI  is  a  symbolic  representation  of  the  classical 
electro-mechanical  Horizontal  Situation  Indicator  (HSI) 
with  greater  inherent  flexibility.  An  illustration  of  the 
EHSI  display  and  a  brief  description  of  the  symbology  is 
shown  in  Figure  38. 

Around  the  periphery  of  the  EHSI  format  are  several 
button  selectable  options.  These  include: 

WYPT  -  Selection  of  this  option  puts  great  circle 
steering  to  the  selected  waypoint  (3  in  the  example)  on  the 
HUD  heading  scale. 

4  -  The  up  arrow  increments  the  waypoint  number  and 

the  down  arrow  decrements  the  waypoint  number.  The 
selected  waypoint  number  is  displayed  between  the  two 
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—  Tha  HOTAS  Cursor  Must  Bs  Positionsd  Within  ths 
Radar  Vidao. 


-  Whan  tha  TDC  is  Daprassad  and  Hald,  tha  HOTAS 
Cursor  is  Rapiacad  by  a  Slawabla  In-Vidao  Cursor. 

—  Position  tha  ln*Vidao  Cursor  Ovar  tha  Targat  and 
Raieasa  tha  TDC. 


—  Tha  in-Vidao  Cursor  is  Raplaead  by  a  Stabllizad 
Cue  indicating  tha  Location  of  tha  Designation. 

—  If  Any  Other  Source  (FUR,  Visual,  Navigation  Data) 
is  Used  to  Designate  tha  Targat,  a  Stabilized  Cue 
Will  Also  Appear  on  tha  Radar  Vidao. 


Not*:  Tlu*  mtthod  or  r«dv  designation  i»  uMO  in  MAS  and  EXPi  mod**. 


Figure  37  RAM/ACE  Radar  Designation  Procedure 

arrows.  There  are  ten  (0-9)  waypoint  numbers  available  and 
three  mark  locations  (Ml,  M2,  M3)  that  can  also  be  used  as 
waypoints.  The  waypoints  can  also  be  incremented  via  the 
waypoint  step  button  on  the  front  seat  throttle  or  aft  seat 
hand  controllers. 

NAVDSG  -  When  "NAVDSG"  is  depressed,  a  designation  is 
performed  on  the  selected  waypoint  number  and  all  available 
sensors  are  slaved  to  the  designation  line-of -sight. 

HDD  -  Selects  the  Head  Down  Display  used  for  Terrain 
Following  flight. 


« 


« 


▼ 

▼ 

0 


9 


•Ml 

Mr 

-> 


«  ■ 
r  « 


TOC  uttgiuntni  Mttftutw* 

Wtypow*  tiluMton  lymool 

pot*)  iitu*lw>  tympoi  •***  ippipcM 
tM  ««yp«M|  titiMtiOO  i»«W 

WtifPOMl  bortAg  potMtt 
Ontguoittf  potAi  tww|  ponMM 
tuf  li  pwnlpf  1*1* 

»*M«sKi*Agtiumnogo  M  UOCIM  wtypptW 

Ul»'  to«  *o»  nfyiMtc  MM'*# 

Of«WMp  UK*  PO*M* 

Moa»m  mod  M 
HMcVtg  WMCI M  MAM* 

C«OfM  IMP  "UMPAW 
Cokmkmmhm* 

AtfC'ill  m*#M<*  M  tNUMlMM 

CotUM  Pi  f*0*  lOOlpHM 
C«M«  •»  *UM  (OOtpAM 


Figure  38  RAM/ACE  EHSI  Display 


«.*,  «-t  <Ll  «  *  -•  «  A  «-*>  .V." 


n  ns  <  »w  t-*j  \--v  ■  jm  i-*M  J-mi. 


£  r»  vn  m  Iffi la M 


SENSORS  -  When  this  option  is  selected,  the  centers  of 
the  Radar  (R)  and  FLXR  (F)  footprints  are  displayed  on  the 
EHSI. 

CRS  HOG  -  If  waypoint  steering  (WYPT)  is  selected  and 
the  up  and  down  arrow  associated  with  the  NCRSN  option  is 
also  selected,  the  course  line  appears  and  a  digital 
readout  of  waypoint  course  is  displayed  on  the  lower  right 
of  the  EHSI.  The  course  line  rotates  about  the  waypoint 
symbol  on  the  EHSI  as  the  course  changes.  If  the  button 
associated  with  the  "CRS"  option  is  depressed,  "CRS" 
changes  to  NHDGN.  Now  the  up  and  down  arrows  rotate  the 
heading  select  bug  around  the  periphery  of  the  compass  rose 
and  also  change  the  Heading  Select  (HS.RL)  digital  readout 
on  the  lower  left  of  the  EHSI. 

SCL/XXX  -  Depressing  the  button  above  the  range  scale 
option  decreases  the  EHSI  range  scale  one  increment.  The 
range  scales  available  are  160,  SO,  40,  20,  and  10  nautical 
miles.  The  range  scales  are  from  the  center  of  the 
aircraft  symbol  to  the  inner  diameter  of  the  compass  rose. 
When  the  range  scale  is  at  10  NM,  one  more  depression  of 
the  scale  button  scrolls  the  range  scale  back  to  160  NM. 

MKX  -  There  are  three  mark  locations  available  in  the 
navigation  computer,  HK1,  MK2,  and  KK3.  Depression  of  the 
button  above  MKl  will  store  the  coordinates  of  the 
designated  point,  if  designated,  or  perform  and  overfly 
mark  if  undesignated.  Also,  the  mark  location  number  will 


automatically  increment  to  MK2.  If  all  three  mark 
locations  are  full  and  the  mark  button  i.s  depressed,  the 
new  location  overwrites  the  previously  stored  location. 

DATA  -  When  "DATA*  is  selected,  the  EHSI  data  sublevel 
is  selected  which  is  illustrated  in  Figure  39. 

Depression  of  the  data  option  calls  up  the  data 
display  as  shown.  ttA/C"  is  automatically  the  selected  data 
option  when  the  data  button  is  depressed.  Whenever  the 
data  sublevel  is  selected,  the  compass  rose,  pointers, 
waypoint  and  TACAN  symbols  and  non-data-associated  options 
are  removed  from  the  display.  Upon  depression  of  HSI,  the 
display  returns  to  normal  HSI  display.  To  review  stored 
waypoint  data,  the  *WYPTN  option  on  the  EHSI  sublevel 
display  should  be  selected.  An  illustration  of  the 
waypoint  sublevel  display  is  shown  in  Figure  40. 

The  waypoint  number  is  incremented  or  decremented 
using  the  buttons  next  to  the  up  and  down  arrows.  Data  for 
waypoints  0  through  9  plus  Mark  1,  Mark  2,  and  Mark  3  can 
be  displayed.  Changing  the  waypoint  number  on  the  data 
sublevel  display  does  not  change  the  waypoint  number 
selected  for  steering.  To  enter  data  for  a  waypoint,  the 
UFC  option  button  is  depressed  and  the  latitude,  longitude , 
and  elevation  data  are  entered  via  the  up-front  control. 

The  HSI  button  is  used  to  return  to  the  top  level  EHSI 
display. 
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Figure  39  RAM/ACE  EHSI  Data  Sublevel,  Aircraft  Option 
Selected 

SAR  POP-UP  MECHANIZATION 

For  the  radar  to  generate  a  SAR  map  the  radar  needs 
line-of-sight  (LOS)  to  the  target  area.  If  the  ingressing 
aircraft  is  terrain  following  to  avoid  threats  a  pop-up 
maneuver  is  required  to  get  radar  LOS.  To  help  the  pilot 
perform  the  pop-up  maneuver  and  make  sure  that  target  LOS 
is  achieved,  when  the  pilot  commands  pop-up  using  the  front 
seat  throttle  switch,  the  TF  cue  on  the  HUD  reverts  to  a 


219 


Figure  40  RAM/ ACE  EHSI  Data  Sublevel,  Waypoint  Option 
Selected 

command  pop-up  symbol,  which  displays  longitudinal  commands 
only,  as  shown  in  Figure  41.  Upon  selection  of  the  pop-up 
mode,  all  TF  modes  are  turned  off.  The  pilot's  task  is  to 
manually  fly  the  velocity  vector  to  the  pop-cue.  If  the 
steering  cue  is  followed,  the  aircraft  will  climb  until  a 
3°  depression  angle  to  the  target  is  achieved.  The 
commanded  altitude  will  stay  constant  until  a  SAR  map  is 
formed.  At  that  time,  the  pilot  will  have  to  cancel  the 
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Figure  41  RAM/ ACE  Pop-Up  Command  Symbology  on  the  HUD 


pop-up  mode  and  use  the  UFC  to  re-engage  a  TF  mode. 

The  aircrew  has  to  determine  when  to  command  the  pop¬ 
up  cue  as  there  is  no  automated  coupling  to  target  range  or 
offset  angle.  Also,  the  aircraft  does  not  have  to  be  level 
when  the  SAR  map  is  taken.  The  limiting  factors  are; 
a)  Less  than  or  equal  to  4G* s  longitudinal 
b}  No  asymmetrical  G  limit 
c)  Between  8°  and  60°  heading  offset. 


Il *  lit  UvVv  A i  JvU  /  JGuOuviJtfQOoCKJvVViWyVu 


SENSOR  IMAGERY  INTERPRETATION 


The  two  imaging  sensors  aboard  the  ABL  are  the  SAR 
radar  and  the  FLIR.  Both  sensors  provide  imagery  to  the 
operator  in  a  television-like  video  format.  The  SAR 
imagery  is  substantially  different  than  normal 
monochromatic  video  imagery  from  cameras.  The  FLIR  imagery 
looks  much  like  television  video  but  is  generated  by  a 
substantially  different  mechanism.  The  following  sections 
provide  a  brief  description  of  the  sensor  imagery. 

SAR  Imagery 

The  displayed  imagery  from  the  SAR  differs  from  normal 
video  pictures  in  that  it  presents  a  bird's  eye  view  of  the 
area  being  mapped  by  the  radar  regardless  of  the  line  of 
sight  depression  angle  from  aircraft  to  target.  In  normal 
video,  the  shades  of  gray  are  a  measure  of  the  amount  of 
light  reflected  from  an  object  to  the  camera.  In  the  SAR 
imagery,  the  brighter  areas  are  those  of  higher  radar 
return,  while  dark  areas  are  those  of  little  or  no  return. 
When  something  blocks  the  radar  beam  or  return,  a  dark 
area,  or  "shadow"  appears.  Figure  42  presents  a  sample  of 
high  resolution  SAR  imagery.  Shown  in  this  figure  are 
areas  of  varying  return,  radar  "shadows",  and  bright  spots 
showing  hard  targets  (large  returns).  Since  SAR  imagery  is 
very  different  from  what  most  aircrews  are  used  to  seeing, 
sample  imagery  of  test  targets  will  be  discussed  during  the 
sensor  training  session.  Figure  43  presents  a  comparison 


Figure  42  Representative  SAR  Imagery 


of  a  paper  map,  a  SAR  map,  and  an  aerial  photograph  of  the 
same  ground  area. 


FLIR  Imagery 

FLIR  imagery  differs  from  normal  video  in  that  the 
gray  shade  are  a  measure  of  the  apparent  temperature  of  an 


Figure  43  Imagery  Comparison 


object  or  area  in  the  scene.  The  FLIR  sensor  converts  the 
infrared  energy  emitted  by  the  scene  to  the  visible 
spectrum.  Most  FLIR  systems  have  the  option  to  set  the 
video  to  show  the  hottest  areas  as  bright  spots  and  the 
coolest  areas  as  dark  (white  hot) ,  or  just  the  reverse 
(black  hot) .  For  either  method,  the  differing  levels  of 
brightness  on  the  imagery  indicate  the  relative  temperature 
of  that  area  of  the  scene.  The  white  hot  presentation 
usually  provides  a  picture  which  looks  very  similar  to 
normal  television  video.  The  black  hot  presentation,  which 
is  used  in  this  simulation,  provides  a  reverse  image 
effect.  Figure  44  presents  some  FLIR  imagery 
representative  of  that  produced  by  second  generation  FLIR 
sensors . 

Since  FLIR  imagery  looks  so  much  like  television 
imagery  and  no  special  interpretation  techniques  will  be 
used  in  the  RAM/ACF  study,  the  FLIR  interpretation  training 
will  be  limited  to  viewing  imagery  during  the  practice 
mission  runs. 

HR  MAVERICK 

The  IIR  Maverick,  as  shown  in  Figure  45,  is  an 
infrarad  guided,  rocket  powered,  air-to-surface  missile. 

It  is  operated  in  a  launch  and  leave  attack  mode.  The 
primary  use  of  the  missile  is  to  provide  high  probability 
of  kill  against  small  hard  targets.  The  missile  is  98 
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Figure  45  Imaging  Infrared  Maverick 


"A/G  WPN"  option  on  either  the  MENU  or  FLIR  formats  can  be 
used  to  select  the  HR  Maverick  display. 

The  long  horizontal  and  vertical  lines  on  the  IIR 
Maverick  display  represent  the  missile  boresight  lines. 

The  opening  is  called  the  track  gate.  The  three  horizontal 
lines  centered  in  azimuth,  below  the  display  center 
represent  -10°,  -20°,  and  -30°  in  elevation.  The  RAM/ACE 
IIR  Maverick  model  allows  the  missile's  head  to  be  slewed 
+30°  in  elevation  and  +40°  in  azimuth.  The  TDC,  in  the  no 
action  position,  is  used  to  slew  the  Maverick  head.  Each 
Maverick  must  be  uncaged  prior  to  slewing  or  lock-on.  The 
cross  at  the  center  of  the  display  in  Figure  46  is  the 
Maverick  head  indicator  which  moves  around  the  display 


MAVERICK  READY 


Figure  46  HR  Maverick  Display  . 


The  MENU  or  FLIR  display  can  be  selected  directly  from 
the  Maverick  format  using  the  button  options.  Also,  the 
FLIR/Maverick  display  alternate  switch  can  be  used  to 
rapidly  alternate  between  the  FLIR  and  Maverick  formats. 


ABL  Ratings 


Now  that  you  have  completed  reading  the  description  of 
the  ABL,  please  give  your  best  estimate  of  pilot  workload 
for  this  configuration.  You  may  refer  back  to  any  part  of 
this  document  as  needed.  Ratings  should  be  made  by  placing 
either  a  1  (low),  2  (medium),  or  3  (high)  in  each  of  the 
blank  spaces  below.  Refer  to  page  5  of  this  document  for  a 
refresher  on  the  definition  of  each  of  these  ratings. 

ABL 

Mental  Psychological 

Segment  Time  Load  Effort  Load  Stress  Load 

Penetration  _  _  _ 

Target 

Acquisition  _  _  _ 

Weapon 

Del  ivery  _  _ _  _______ 

Pilot  Name: _ _ 

Unit: _ _ 

Your  thoughts  and  comments  on  this  part  of  the 
exercise  are  welcome.  You  may  use  the  back  of  this  page 
for  any  comments  you  may  have.  After  completion  of  th»s 
page,  remove  it  from  this  book  and  nail  it,  with  ttys  other 
two  rating  sheets,  in  the  envelope  provided.  No:v  continue 
to  the  next  section. 


TF/TA  Instructions 
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The  next  section  contains  a  description  of  an 
automatic  Terrain  Following/Terrain  Avoidance  (TF/TA) 
subsystem.  Consider  it  to  be  added  to  the  ABL 
configuration.  The  TF/TA  has  two  operating  modes:  manual 
TF/TA  and  automatic  TF/TA.  The  manual  TF/TA  mode  provides 
pitch  and  steering  commands ,  but  requires  the  pilot  to 
manually  fly  the  aircraft.  Automatic  TF/TA  is  coupled  to 
the  flight  control  system  and  requires  the  pilot  to  simply 
monitor  the  system  via  the  KUO  or  HDD.  For  the  purpose  of 
this  exercise,  consider  the  TF/TA  system  to  be  in  the 
automatic  mode. 

Once  again,  keep  in  mind  the  BI  mission  previously 
described.  Estimates  will  be  required  for  each  segment 
(Penetration,  Target  Acquisition,  and  Weapon  Delivery)  in 
each  dimension  (Timeload,  Mental  Effort  Load,  and 
Psychological  Stress  Load) . 


AUTOMATIC  TF/TA 


The  automatic  TF/TA  enhancement  has  a  flight 
trajectory  generator  that  is  expanded  to  three  dimensions 
from  the  vertical  plane  limitations  of  TF  flight  only.  The 
system  generates  flight  commands  which  can  be  flown 
manually  by  the  pilot  or  coupled  directly  to  the  flight 
control  system  so  that  the  aircraft  automatically  follows 
the  generated  trajectory.  Data  for  the  TF/TA  algorithms 
are  supplied  from  a  combination  of  sensed  data  (e.g.,  radar 
altimeters,  forward  looking  radars  and  laser  scanners)  and 
stored  data  (e.g..  Digital  Land  Mass  Data).  In  general, 
both  pitch  and  roll  maneuvers  are  used  to  maximize  the 
masking  benefits  afforded  by  the  topography.  The  flight 
path  is  constrained  by  the  mission  route  between  waypoints 
and  maximum  desired  lateral  deviations. 

The  UFC  is  used  to  enable  the  TF/TA  mode  as  discussed 
previously.  Both  automatic  and  manual  TF/TA  modes  are 
available.  In  automatic  TF/TA  modes  the  pilot  simply 
monitors  the  system  via  the  HUD  and  HDD  as  necessary.  In 
manual  TF/TA  mode  the  same  commands  are  displayed  on  the 
HUD  and  HDD  as  in  automatic  TF/TA  mode.  The  HUD  symbology 
for  TF/TA  is  shown  in  Figure  47. 

The  format  shows  the  pilot  a  programmed  tactical 
corridor  which  represents  the  commanded  TF/TA  flightpath. 
The  corridor  is  shown  as  three  connected  boxes.  The  boxes 
are  connected  on  the  bottom  center  with  a  dashed  line.  The 
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Figure  47  TF/TA  HUD  Format,  Programmed  Tactical  Corridor 

corridor  is  roll  stabilized  and  always  provides  the  pilot 
with  flight  path  commands  down  the  corridor  or  back  to  the 
corridor  if  he  strays  away.  On  the  right  side  of  the  HUD 
the  automatic  terrain  following  (ATF)  commanded  altitude  is 
shown  to  allow  the  pilot  to  correlate  commanded  altitude 
versus  actual  altitude.  The  foreground  box  shows  commanded 
course,  which  is  assumed  to  be  in  the  center  of  the  box. 

The  second  and  third  boxes  are  earth-position  stabilized 
over  the  next  two  significant  terrain  features  and/or  pre¬ 
programmed  turn  points.  As  the  aircraft  approaches  it,  the 
second  box  increases  in  size  until  it  corresponds  to  the 
current  (i.e.,  foreground)  box.  At  the  instant  these  two 


boxes  coincide,  the  aircraft  is  passing  the  next  action 
point.  The  third  box  now  becomes  the  second  and  a  new 
third  box  appears  in  the  distance. 

•» 

Switching  from  automatic  to  manual  TF/TA  can  be 
accomplished  rapidly  via  the  paddle  switch  located  on  the 
front  of  the  stick.  At  anytime  the  pilot  can  override  the 
automatic  TF/TA  commands  by  applying  manual  stick  forces  of 
greater  than  2  pounds.  Once  the  stick  is  released  (less 
than  2  pounds) ,  the  system  automatically  reverts  back  to 
automatic  TF/TA. 


TF/TA  Ratings 


Please  give  your  best  estimate  of  pilot  workload  for 
this  configuration.  You  may  refer  back  to  any  part  of  this 
document  as  needed.  Ratings  should  be  made  by  placing 
either  a  1  (low),  2  (medium),  or  3  (high)  in  each  of  the 
blank  spaces  below.  Refer  to  page  5  of  this  document  for  a 
refresher  on  the  definition  of  each  of  these  ratings. 


TF/TA 

Mental  Psychological 

Segment  Time  Load  Effort  Load  Stress  Load 

Penetration  _  _  _ 

Target 

Acquisition  _  _  _ 

Weapon 

Delivery  _  _  _ 

Pilot  Name:  _ 

Unit:  _ 

Your  thoughts  and  comments  on  this  part  of  the 
exercise  are  welcome.  You  may  use  the  back  of  this  page 
for  any  comments  you  may  have.  After  completion  of  this 
page,  remove  it  from  this  book  and  mail  it,  with  the  other 
two  rating  sheets,  in  the  envelope  provided.  Now  continue 
to  the  next  section. 


TTA  Instructions 


This  is  the  last  section  of  this  exercise  and  it 
contains  a  description  of  an  automatic  Terrain/Threat 
Avoidance  (TTA)  subsystem.  Consider  it  to  be  added  to  the 
ABL  configuration  separately  and  independently  of  the  TF/TA 
subsystem.  The  TTA  system  also  has  two  operating  modes: 
manual  and  automatic.  Once  again,  for  the  purpose  of  this 
exercise,  consider  the  TTA  system  to  be  in  the  automatic 
mode. 

Keep  in  mind  the  mission  scenario  described  on  page  3. 
Estimates  will  be  required  for  each  segment  (Penetration, 
Target  Acquisition,  and  Weapon  Delivery)  in  each  dimension 
(Time  Load,  Mental  Effort  Load,  and  Psychological  Stress 
Load) . 


AUTOMATIC  TERRAIN/THREAT  AVOIDANCE 


The  classical  method  of  preparing  to  face  surface-to- 
air  threats  is  to  receive  a  premission  intelligence 
briefing  which  educates  the  aircrews  to  the  types  and  last 
known  positions  of  the  threats.  This  data  is  usually  hours 
if  not  days  old  and  is  highly  vulnerable  to  error  due  to 


the  mobility  of  modern  threat  systems.  During  premission 
planning,  the  pilot  plans  his  route  to  avoid  prebriefed 
threats  and  still  meet  his  Time  Over  Target  (TOT)  and  fuel 
requirements.  Once  airborne,  the  pilot  is  required  to 
monitor  his  threat  display  and  adjust  his  route  to  avoid 
detected  threats  real-time  while  flying  low  and  fast. 

After  returning  to  his  recovery  base,  the  pilot  debriefs 
with  intelligence  personnel  on  his  mission.  This  consists 
of  a  Bomb  Damage  Assessment  (BDA)  and  the  aircrew's  best 
guess  on  when  and  where  threat  detection/sightings  were 
made. 

The  automatic  terrain/threat  avoidance  enhancement 
assumes  a  Data  Transfer  Module  (DTM)  has  been  used.  The 
DTM  contains  the  route  of  flight  and  latest  intelligence 
summaries  for  the  area  of  concern.  The  pilot  transferred 
the  data  stored  on  the  DTM  into  his  aircraft  computers  for 
display  on  cockpit  displays.  Airborne,  data  received  from 
the  aircraft's  threat  sensors  is  automatically  fed  to  the 
onboard  computers  and  a  real-time  threat  data  base  update 
is  accomplished.  Once  loaded,  the  computers  analyze  the 
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threats  and  automatically  optimize  the  route  of  flight  to 
skirt  both  the  newly  detected  threats  and  still  perform 
terrain  avoidance  as  required.  The  new  route  will  be  shown 
on  a  cockpit  display  along  with  the  great  circle  route  for 
comparison.  The  pilot  will  have  the  choice  of  accepting 
this  new  route  or  flying  his  own. 

The  UFC  is  used  to  enable  the  TTA  mode.  Once  enabled, 
two  different  displays  need  to  be  monitored,  the  HDD  and 
threat  displays.  The  HDD  will  show  the  terrain  avoidance 
portion  of  the  enhancement  and  the  threat  display  will  show 
the  threat  avoidance  portion  of  the  enhancement.  Both 
formats  will  illustrate  the  pre-computed  great  circle  path 
(solid  line)  and  the  newly  computed  threat  induced  portion 
(dashed  line) .  An  example  of  the  threat  display  during  TTA 
flight  is  illustrated  in  Figure  48. 

The  pilot  has  the  option  of  reverting  to  manual  flight 
simply  by  hitting  the  paddle  switch  on  the  front  of  the 
stick  which  will  keep  the  aircraft  in  TTA  modes  but  will 
require  hand  flying.  Reselection  of  automatic  TTA  mode  has 
to  be  accomplished  via  the  UFC.  Complete  deselection  of 
TTA  mode  also  has  to  be  accomplished  via  the  UFC.  The  HUD 
display  for  TTA  is  the  same  as  for  TF/TA  except  "ATA**  for 
automatic  TTA  and  "MTAM  for  manual  TTA  is  displayed  on  the 
HUD. 


TTA  Ratings 


Please  give  your  best  estimate  of  pilot  workload  for 
this  configuration.  You  may  refer  back  to  any  part  of  this 
document  as  needed.  Ratings  should  be  made  by  placing 
either  a  1  (low) ,  2  (medium) ,  or  3  (high)  in  each  of  the 
blank  spaces  below.  Refer  to  page  5  of  this  document  for  a 
refresher  on  the  definition  of  each  of  these  ratings. 

TTA 

Mental  Psychological 

Segment  Time  Load  Effort  Load  Stress  Load 

Penetration  _  _  _ 

Target 

Acquisition  _  _  _ 

Weapon 

Delivery  _  _  _ 

Pilot  Name:  _ 

Unit :  _ 

Your  thoughts  and  comments  on  this  part  of  the 
exercise  are  welcome.  You  may  use  the  back  of  this  page 
for  any  comments  you  may  have.  After  completion  of  this 
page,  remove  it  from  this  book  and  mail  it,  with  the  other 
two  rating  sheets,  in  the  envelope  provided.  Please  retain 
this  book  until  you  are  directed  to  return  it.  At  that 
time  you  should  return  it  in  the  large  envelope  provided. 
Thank  you  for  fine  efforts. 
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Table  26  SWAT  Workload  Ratings  (1986) ,  ABL  Configuration 


Table  27  SWAT  Workload  Ratings  (1986),  TF/TA 


Configuration 


Segment 

Sub j  ect 

Group 

Ratings 

Workload 

-  -  — 

TL 

PSL 

Score 

PEN 

1 

Effort 

2 

2 

1 

26.2 

2 

Time 

2 

3 

2 

70.4 

3 

Effort 

2 

3 

2 

65.9 

4 

Stress 

1 

2 

2 

41.8 

5 

Time 

2 

3 

3 

76.3 

6 

Stress 

1 

1 

2 

28.8 

7 

Stress 

2 

2 

3 

72.0 

8 

Stress 

2 

2 

2 

52.2 

TF/TA 

1 

Effort 

2 

2 

2 

40.8 

2 

Time 

1 

1 

1 

0.0 

3 

Effort 

3 

1 

2 

45.2 

4 

Stress 

1 

1 

2 

28.8 

5 

Time 

2 

3 

3 

76.3 

6 

Stress 

2 

2 

2 

52.2 

7 

Stress 

2 

3 

2 

63.1 

8 

Stress 

1 

1 

1 

0.0 

TTA 

1 

Effort 

2 

1 

2 

16.0 

2 

Time 

2 

2 

1 

45.7 

3 

Effort 

2 

3 

2 

65.9 

4 

Stress 

1 

2 

2 

41.8 

5 

Time 

2 

2 

2 

59.9 

6 

Stress 

2 

2 

2 

52.2 

7 

Stress 

1 

1 

1 

0.0 

8 

Stress 

1 

1 

1 

0.0 

Table  28  SWAT  Workload  Ratings  (1986) ,  TTA  Configuration 


Segment 

Subject 

Group 

Ratings 

Workload 

Score 

PSL 

PEN 

1 

Effort 

2 

i 

2 

16.0 

2 

Tima 

2 

2 

2 

59.9 

3 

Effort 

2 

2 

2 

40.8 

4 

Stress 

1 

2 

1 

13.0 

5 

Time 

2 

2 

2 

59.9 

6 

Stress 

1 

1 

2 

28.8 

7 

Stress 

1 

1 

3 

48.5 

8 

Stress 

2 

2 

2 

52.2 

TF/TA 

1 

Effort 

2 

2 

2 

40.8 

2 

Time 

1 

1 

1 

0.0 

3 

Effort 

3 

1 

2 

45.2 

4 

Stress 

1 

1 

2 

28.8 

5 

Time 

2 

2 

2 

59.9 

6 

Stress 

2 

2 

2 

52.2 

7 

Stress 

2 

3 

2 

63.1 

8 

Stress 

2 

2 

2 

52.2 

TTA 

1 

Effort 

2 

1 

1 

1.4 

2 

Time 

2 

3 

3 

76.3 

3 

Effort 

2 

2 

2 

40.8 

4 

Stress 

1 

2 

2 

41.8 

5 

Time 

2 

2 

2 

59.9 

6 

Stress 

2 

2 

2 

52.2 

7 

Stress 

1 

1 

1 

0.0 

8 

Stress 

1 

2 

2 

41.8 
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This  Appendix  provides  detailed  information  about  the 
twelve  different  task  categories  used  in  the  discrete  event 
simulation  models.  This  information  is  presented  in  four 
tables.  The  first  of  these.  Table  29,  presents  all  perti¬ 
nent  information  by  task  category.  It  begins  with  the 
category  title  and  an  indication  of  whether  the  task  is 
repetitive  (R)  or  non-repetitive  (N) .  TYPE  is  a  number 
assigned  to  each  category  for  identification  purposes. 

PRIORITY  refers  to  the  priority  initially  assigned  to 
that  task  category,  as  described  in  Chapter  IV.  The  task 
inter-arrival  time  and  service  time  distributions  are  also 
listed,  with  appropriate  parameter  values.  The  mean  values 
of  each  of  these  distributions  presented  in  Table  29  apply 
to  the  Penetration  Segment  of  the  Advanced  Baseline  (ABL) 
configuration.  The  standard  deviations  apply  to  all 
segments  and  all  configurations.  The  Multiple  Resource 
Model  requirements  are  also  listed  along  with  numerical 
codes  (1  or  2)  to  designate  which  dimension  of  each 
resource  is  required.  Finally,  a  brief  verbal  description 
of  each  task  is  presented. 

Table  30  lists  the  mean  values  of  the  inter-arrival 
time  and  service  time  distributions  for  each  flight  segment 
and  all  cockpit  configurations.  Chapter  IV  describes  in 
detail  how  these  values  were  developed  from  those  found  in 


Table  29. 
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Table  29  Pilot  Task  Summary 


TASKS 

FLY  AIRCRAFT 

NAVIGATE 

ASSESS 

ENVIRONMENT 

TASK  TYPE 

1 

2 

3 

TASK  PRIORITY 

3 

2 

2 

INTER-ARRIVAL 

TIME 

NORM(9 . 5, 1) 

NORM (90,5) 

NORM (55, 1. 5) 

SERVICE  TIME 

EXP (1. 509) 

NORM (3,0.5) 

NORM (3,0.7) 

MULTIPLE 

RESOURCE  MODEL 

MODALITY 

VISUAL  (1) 

VISUAL  (1) 

VISUAL  (1) 

CODE 

SPATIAL  (2) 

VERBAL  (1) 

SPATIAL  (2) 

STAGE 

RESPOND  (2) 

ENCODE  (1) 

ENCODE  (1) 

REPETITIVE/ 

NONREPETITIVE 

R 

R 

R 

Table  29  Pilot  Task  Summary  (Continued) 


TASKS 

ASSESS 

MISSION  PLAN 

MONITOR/ASSESS 

AIRCRAFT 

SYSTEMS 

CONFIGURE 

AIRCRAFT 

SYSTEMS 

TASK  TYPE 

4 

5 

6 

TASK  PRIORITY 

1 

2 

3 

INTER-ARRIVAL 

TIME 

NORM(250, 20) 

NORM (10, 1) 

NORM (35, 3) 

SERVICE  TIME 

NORM (5,1.2) 

NORM (1,0.2) 

NORM (1.9, .45) 

MULTIPLE 
RESOURCE  MODEI 

MODALITY 

VISUAL  (1) 

VISUAL  (1) 

VISUAL  (1) 

CODE 

VERBAL  (1) 

VERBAL  (1) 

SPATIAL  (2) 

STAGE 

ENCODE  (1) 

ENCODE  (1) 

RESPOND  (2) 

REPETITIVE/ 

NONREPETITIVE 


R 


R 


R 


Table  29  Pilot  Task  Summary  (Continued) 


TASKS 

WAYPOINT 

ARRIVAL 

MANEUVER 

AIRCRAFT 

DEFENSIVE 

MANEUVER 

TASK  TYPE 

7 

8 

9 

TASK  PRIORITY 

3 

3 

3 

INTER-ARRIVAL 

TIME 

ONCE  ONLY 

ONCE  ONLY 

EXP (550) 

SERVICE  TIME 

NORM (5,1) 

NORM (9,2) 

NORM (10, 2) 

MULTIPLE 
RESOURCE  MODEL 

MODALITY 

VISUAL  (1) 

VISUAL  (1) 

VISUAL  (1) 

CODE 

SPATIAL  (2) 

SPATIAL  (2) 

SPATIAL  (2) 

STAGE 

ENCODE  (1) 

RESPOND  (2) 

RESPOND  (2) 

REPETITIVE/ 

NONREPETITIVE 

N 

N 

R 

Table  29  Pilot  Task  Summary  (Continued) 
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TASK  TYPE 

TASK  PRIORITY 

INTER-ARRIVAL 

TIME 

SERVICE  TIME 

MULTIPLE 
RESOURCE  MODE: 

MODALITY 

CODE 

STAGE 

REPETITIVE/ 

NONREPETITIVE 


TASKS  | 

ACQUIRE 

TARGET 

ACTIVATE 

OFFENSIVE 

SYSTEMS 

COMMUNICATIONS 

10 

11 

12 

3 

3 

2 

NORM(16.95,4) 

ONCE  ONLY 

ONCE  ONLY 

NORM (4/0.9) 

NORM (5,1) 

NORM (10, 2) 

VISUAL  (1) 

VISUAL  (1) 

AUDITORY  (2) 

SPATIAL  (2) 

SPATIAL  (2) 

VERBAL  (1) 

ENCODE  (1) 

RESPOND  (2) 

ENCODE  (2) 

R 

N 

N 

Table  30  Task/Segment  Summary,  All  Configurations,  Mean 
Inter-arrival  and  Service  Times 


SEGMENT 

PENETRATION 

TARGET 

ACQUISITION 

WEAPON 

DELIVERY 

TASK 

VA 

VA 

14c 

FLY 

AIRCRAFT 

9.5 

1.509 

11.4 

1.509 

4.56 

1.509 

NAVIGATE 

90.0 

3.0 

67,5 

3.0 

67.5 

3.0 

ASSESS 

ENVIRONMENT 

55.0 

3.0 

18.0 

3.0 

18.0 

3.0 

ASSESS 

MISSION 

PLAN 

250.0 

5.0 

150.0 

5.0 

150.0 

5.0 

MONITOR/ 

ASSESS 

AIRCRAFT 

SYSTEMS 

10.0 

1.0 

6.0 

1.0 

6.0 

1.0 

CONFIGURE 

AIRCRAFT 

SYSTEMS 

35.0 

3.0 

24.5 

D 

24.5 

3.0 

WAYPOINT 

ARRIVAL 

300.0 

sJ  •  u 

MANEUVER 

AIRCRAFT 

300.0 

9.0 

— 

■ 

— - 

— 

DEFENSIVE 

MANEUVER 

550.0 

10.0 

357.5 

10.0 

357.5 

10.0 

ACQUIRE 

TARGET 

— 

— 

16.95 

4.0 

— — 

— - 

ACTUATE 

OFFENSIVE 

SYSTEM 

— 

— 

— — 

— 

33.9 

B 

COMMUNI¬ 

CATIONS 

600.0 

10.0 

— 

— 

— 

— 

1/X  -  Mean  inter-arrival  time 
1/m.  -  Mean  service  time 


APPENDIX  F 


SIMULATION  PROGRAMS 


The  first  part  of  this  appendix  lists  the  serial  processing 
simulation  program  while  the  second  part  lists  the  parallel 
processing  simulation  program. 


SERIAL  PROCESSING  MODEL: 


PREAMBLE  LAST  COLUMN  IS  72  " 

•'PROGRAM  SSABL  -  ENTIRE  MISSION,  25  REPLICATIONS 
EVENT  NOTICES  INCLUDE  END. SIM,  DATA. RESET, 

A. ARRIVAL,  B. ARRIVAL,  C. ARRIVAL, 

D. ARRIVAL,  E. ARRIVAL,  F. ARRIVAL, 

G. ARRIVAL,  H. ARRIVAL,  I. ARRIVAL, 

J. ARRIVAL,  K. ARRIVAL,  L. ARRIVAL, 

PEN.ACQ,  ACQ.WPN 
EVERY  E.O.S  HAS  A  JOB 
THE  SYSTEM  OWNS  A  QUEUE,  AND  A  WORK 
PERMANENT  ENTITIES 

EVERY  PILOT  HAS  A  BUSY 
TEMPORARY  ENTITIES 

EVERY  TASK  HAS  AN  A. TIME,  A  TYPE,  A  PRIOR, 

A  SERV . TIME  AND  MAY  BELONG  TO  THE  QUEUE,  AND 
THE  WORK 

DEFINE  SECONDS  TO  MEAN  UNITS 

DEFINE  SIM. TIME,  A. TIME,  INT.4,  INT.5,  INT.6,  INT.7, 

INT.8,  INT.9,  P. IDLE,  A. IDLE,  W.IDLE,  INT.l,  INT.2, 
INT.3,  SERV. TIME  AS  REAL  VARIABLES 
DEFINE  BUSY,  TYPE,  PRIOR,  N,  INTER,  NUM. ARRIVE,  JOB, 

NCOUNT ,  POSSIBLE. INT  AS  INTEGER  VARIABLES 
DEFINE  Pl.ARR,  P2.ARR,  PI. SERV,  P2.SERV  AS  1-DIM  VARIABLES 
DEFINE  QUEUE  AS  A  SET  RANKED  BY  PRIOR 
DEFINE  PREEMPT  AS  A  ROUTINE  GIVEN  1  ARGUMENT 
TALLY  INT. TOT  AS  THE  SUM  OF  INTER 
TALLY  ARR.TOT  AS  THE  SUM  OF  NUM. ARRIVE 
TALLY  POS.TOT  AS  THE  SUM  OF  POSSIBLE. INT 
ACCUMULATE  AV.BUSY  AS  THE  MEAN  OF  BUSY 
' '  THE  FOLLOWING  STATEMENTS  ARE  USED  TO  COLLECT  FINAL 
• '  STATISTICS 

TALLY  PEN. MEAN  AS  THE  MEAN,  PEN. VAR  AS  THE  VARIANCE  OF 
P. IDLE 

TALLY  ACQ.MEAN  AS  THE  KEAN,  ACQ.VAR  AS  THE  VARIANCE  OF 
A. IDLE 

TALLY  WPN.MEAN  AS  THE  MEAN,  WPN.VAR  AS  THE  VARIANCE  OF 
W.IDLE 


TALLY  PEN1.INT  AS  THE  MEAN,  PEN 2 . INT  AS  THE  VARIANCE  OF 
INT.l 

TALLY  ACQ1.INT  AS  THE  MEAN,  ACQ2.INT  AS  THE  VARIANCE  OF 
INT.  2 

TALLY  WPN1.INT  AS  THE  MEAN,  WPN2.INT  AS  THE  VARIANCE  OF 
INT.  3 

TALLY  PEN 3 . INT  AS  THE  MEAN,  PEN4.INT  AS  THE  VARIANCE  OF 
INT.  4 

TALLY  ACQ3.INT  AS  THE  MEAN,  ACQ4.INT  AS  THE  VARIANCE  OF 
INT.  5 

TALLY  WPN3.INT  AS  THE  MEAN,  WPN4.INT  AS  THE  VARIANCE  OF 
INT.  6 

TALLY  PENS. INT  AS  THE  MEAN,  PEN 6 . INT  AS  THE  VARIANCE  OF 
INT.  7 

TALLY  ACQ5.INT  AS  THE  MEAN,  ACQ6.INT  AS  THE  VARIANCE  OF 
INT.  8 

TALLY  WPN5.INT  AS  THE  MEAN,  WPN6.INT  AS  THE  VARIANCE  OF 
INT. 9 

END 

MAIN 

DEFINE  II,  J,  REPS  AS  INTEGER  VARIABLES 
READ  SIM. TIME,  RESET. TIME 
START  NEW  CARD 
READ  N,  REPS 

PRINT  3  LINES  WITH  N,  REPS  THUS 

THIS  EXPERIMENT  MODELS  A  PILOT  IN  AN  AIR-TO-GROUND  MISSION 
***  DIFFERENT  TASKS  ARE  USED  TO  LOAD  THE  PILOT. 

***  REPLICATIONS  OF  EACH  MISSION  ARE  ACCOMPLISHED. 

SKIP  2  LINES 

RESERVE  P1.ARR(*)  AS  N,  P2.ARR(*)  AS  N, 

PI . SERV ( * )  AS  N,  P2 . SERV ( * )  AS  N 
PRINT  2  LINES  THUS 

TASK  NUMBER  ARRIVAL  RATE  SERVICE  RATE 

PARAM  1  PARAM  2  PARAM  1  PARAM  2 
FOR  II  -  1  TO  N,  DO 
START  NEW  CARD 

READ  Pl.ARR(II),  P2.ARR(II),  Pl.SERV(II) ,  P2.SERV(II) 
PRINT  1  LINE  WITH  II,  PI.ARR(II) ,  P2.ARR(II), 

Pl.SERV(II) ,  P2. SERV (II)  THUS 

****  ft.****#  *.*****  *.*«**#  *.***** 

LOOP 

SKIP  2  LINES 

PRINT  2  LINES  WITH  SIM. TIME,  RESET. TIME  THUS 
TOTAL  SIMULATION  TIME  •  ******  SECONDS 

DATA  RESET  TIME  «  ft***.*  SECONDS 

SKIP  2  LINES 
LET  N. PILOT  -  1 
CREATE  EVERY  PILOT 
LET  PILOT  -  1 


FOR  J  *  1  TO  REPS 
DO 

SCHEDULE  AH  A. ARRIVAL  NOW 
SCHEDULE  A  B. ARRIVAL  NOW 
SCHEDULE  A  C. ARRIVAL  NOW 
SCHEDULE  A  D. ARRIVAL  NOW 
SCHEDULE  AN  E. ARRIVAL  NOW 
SCHEDULE  A  F. ARRIVAL  NOW 
SCHEDULE  A  G. ARRIVAL  IN  300.  SECONDS 
SCHEDULE  AN  I. ARRIVAL  NOW 
SCHEDULE  A  PEN.ACQ  IN  500  SECONDS 
SCHEDULE  A  ACQ. WPN  IN  545  SECONDS 
SCHEDULE  A  L. ARRIVAL  IN  250.  SECONDS 
SCHEDULE  A  DATA. RESET  IN  RESET. TIME  SECONDS 
SCHEDULE  AN  END. SIM  IN  SIM. TIME  SECONDS 
START  SIMULATION 
LOOP 


PRINT  2  LINES  THUS 

IDLE  TIME  RATIO  SUMMARY 

PHASE  MEAN  STANDARD  DEVIATION 

PRINT  3  LINES  WITH  PEN. MEAN,  SQRT . F ( PEN . VAR/REPS ) , 

ACQ.MEAN,  SQRT. F(ACQ. VAR/REPS ) , 

WPN . MEAN ,  SQRT. F(WPN. VAR/REPS)  THUS 


PENETRATION  *.****  ******** 

TARGET  ACQUISITION  ******  **.***** 

WEAPON  DELIVERY  ******  ******** 

SKIP  2  LINES 
PRINT  2  LINES  THUS 

TASK  INTERRUPTION  RATIO  SUMMARY 
PHASE  MEAN  STANDARD  DEVIATION 

PRINT  9  LINES  WITH  PEN1.INT,  SQRT . F ( PEN2 . INT/REPS ) , 

ACQ1 . INT ,  SQRT . F (ACQ2 . INT/REPS ) , 

WPN1 . INT ,  SQRT . F (WPN2 . INT/REPS ) , 

PEN3 . INT ,  SQRT . F ( PEN4 . INT/REPS ) , 

ACQ3 . INT ,  SQRT . F { ACQ4 . INT/REPS ) , 

WPN3 . INT ,  SQRT . F (WPN4 . INT/REPS ) , 
PEN5.INT,  SQRT.F(PEN6. INT/REPS) , 
ACQ5.INT,  SQRT. F(ACQ6. INT/REPS) , 
WPN5.INT,  SQRT. F(WPN6. INT/REPS)  THUS 


PENETRATION 

****** 

**  * ***** 

TARGET  ACQUISITION 

****** 

******** 

WEAPON  DELIVERY 

****** 

******** 

PEN  (NEW) 

****** 

******** 

ACQ  (NEW) 

*  *  **** 

******** 

WPN  (NEW) 

****** 

******** 

PEN  (POSSIBLE) 

****** 

******** 

ACQ  (POSSIBLE) 

****** 

******** 

WPN  (POSSIBLE) 

****** 

******** 

CALL  SNAP . R 

STOP 

END 


EVENT  A. ARRIVAL  SAVING  THE  EVENT  NOTICE 

' • FLY  AIRCRAFT  TASK 

DEFINE  NEW1  AS  AN  INTEGER  VARIABLE 

CREATE  A  TASK  CALLED  NEW1 

LET  NUM. ARRIVE  -  1 

LET  A. TIME (NEW1)  «  TIME.V 

LET  TYPE(NEWl)  -  1 

LET  SERV.TIME (NEW1)  »  EXPONENTIAL. F (PI. SERV(l) ,  1) 

LET  PRIOR (NEW1)  -  3 
CALL  PREEMPT  GIVEN  NEW1 
RESCHEDULE  THIS  A. ARRIVAL  IN 

NORMAL.  F(P1.ARR(1)  ,  P2.ARR(1),1)  SECONDS 
RETURN 
END 

EVENT  B. ARRIVAL  SAVING  THE  EVENT  NOTICE 
• 'NAVIGATE  TASK 

DEFINE  NEW 2  AS  AN  INTEGER  VARIABLE 

CREATE  A  TASK  CALLED  NEW2 

LET  NUM. ARRIVE  »  1 

LET  A. TIME (NEW 2 )  -  TIME.V 

LET  TYPE(NEW2)  -  2 

LET  SERV.TIME (NEW 2 )  - 

NORMAL. F( PI. SERV(2) ,  P2.SERV(2) ,  2) 

LET  PRIOR (NEW2)  -  2 
CALL  PREEMPT  GIVEN  NEW2 
RESCHEDULE  THIS  B. ARRIVAL  IN 

NORMAL.  F  (PI.  ARR(2),  P2.ARR(2),  2)  SECONDS 
RETURN 
END 

EVENT  C. ARRIVAL  SAVING  THE  EVENT  NOTICE 

"ASSESS  ENVIRONMENT  TASK 

DEFINE  NEW3  AS  AN  INTEGER  VARIABLE 

CREATE  A  TASK  CALLED  NEW3 

LET  NUM. ARRIVE  «  1 

LET  A. TIME (NEW 3 )  -  TIME.V 

LET  TYPE (NEWS)  -  3 

LET  SERV.TIME (NEWS)  - 

NORMAL. F (PI. SERV(3 ) ,  P2.SERV(3),  3) 

LET  PRIOR (NEW3)  -  2 
CALL  PREEMPT  GIVEN  NEW3 
RESCHEDULE  THIS  C. ARRIVAL  IN 

NORMAL. F ( PI . ARR ( 3 ) ,  P2.ARR(3),  3)  SECONDS 
RETURN 
END 

EVENT  D. ARRIVAL  SAVING  THE  EVENT  NOTICE 

'  'ASSESS  MISSION  PLAN  TASK 

DEFINE  NEW4  AS  AN  INTEGER  VARIABLE 

CREATE  A  TASK  CALLED  NEW4 

LET  NUM. ARRIVE  -  1 

LET  A.TIME(NEW4)  -  TIME.V 

LET  TYPE(NEW4)  -  4 

LET  SERV.TIME (NEW4)  -  NORMAL. F (PI. SERV(4) ,  P2.S£RV(4),  4) 


LET  PRIOR  (NEW4)  »  I 
CALL  PREEMPT  GIVEN  NEW4 
RESCHEDULE  THIS  D. ARRIVAL  IN 

NORMAL. F(PX.ARR (4) ,  P2.ARR(4) ,  4)  SECONDS 
RETURN 
END  ' 

EVENT  E. ARRIVAL  SAVING  THE  EVENT  NOTICE 

1 'MONITOR/ASSESS  AIRCRAFT  SYSTEMS  TASK 

DEFINE  NEW 5  AS  AN  INTEGER  VARIABLE 

CREATE  A  TASK  CALLED  NEW5 

LET  NUM. ARRIVE  -  1 

LET  A. TIME (NEW5)  «  TIME.V 

LET  TYPE(NEW5)  -  5 

LET  SERV. TIME (NEWS)  -  NORMAL.F(Pl.SERV(5) ,  P2.SERV(5) ,  5) 
LET  PRIOR (NEW5)  -  2 
CALL  PREEMPT  GIVEN  NEW5 
RESCHEDULE  THIS  E. ARRIVAL  IN 

NORMAL. F (PI. ARR(5) ,  P2.ARR(5),  5)  SECONDS 
RETURN 
END 

EVENT  F. ARRIVAL  SAVING  THE  EVENT  NOTICE 

• 1 CONFIGURE  AIRCRAFT  SYSTEMS  TASK 

DEFINE  NEWS  AS  AN  INTEGER  VARIABLE 

CREATE  A  TASK  CALLED  NEWS 

LET  NUM. ARRIVE  -  1 

LET  A.TIME(NEW6)  -  TIME.V 

LET  TYPE(NEW6)  -  6 

LET  SERV. TIME (NEWS)  -  NORMAL. F (PI. SERV (6) ,  P2.SERV(6) ,  S) 
LET  PRIOR (NEWS)  “  3 
CALL  PREEMPT  GIVEN  NEWS 
RESCHEDULE  THIS  F. ARRIVAL  IN 

NORMAL. F<Xl  P2.ARR(S) ,  6)  SECONDS 
RETURN 
END 

EVENT  G. ARRIVAL  SAVING  THE  EVENT  NOTICE 
1 'WAYPOINT  ARRIVAL  TASK 
OEFINE  NEW?  AS  AN  INTEGER  VARIABLE 
CREATE  A  TASK  CALLED  NEW? 

LET  NUM. ARRIVE  -  1 

LET  A. TIME (NEW?)  -  TIME.V 

LET  TYPE (NEW?)  -  7 

LET  SERV. TIME (NEW?)  -  NORMAL. F(P1.SERV{ 7) ,  P2.SERV(7),  9) 
LET  PRIOR (NEW7)  -  3 
CALL  PREEMPT  GIVEN  NEW? 

RETURN 

END 

EVENT  H. ARRIVAL  SAVING  THE  EVENT  NOTICE 

"MANEUVER  AIRCRAFT  TASK 

DEFINE  NEW8  AS  AN  INTEGER  VARIABLE 

CREATE  A  TASK  CALLED  NEWS 

LET  NUM. ARRIVE  -  1 

LET  A.TIME(NEW8)  *  TIME.V 


LET  TYPE(NEW8)  -  8 

LET  SERV . TIME (NEWS )  *  NORMAL. F( PI. SERV (8) ,  P2.SERV(8) ,  9) 

LET  PRIOR (NEW8)  -  3 

CALL  PREEMPT  GIVEN  NEW8 

RETURN 

END 

EVENT  I. ARRIVAL  SAVING  THE  EVENT  NOTICE 

' 'DEFENSIVE  MANEUVER  TASK 

DEFINE  NEW9  AS  AN  INTEGER  VARIABLE 

CREATE  A  TASK  CALLED  NEW9 

LET  NUM. ARRIVE  -  1 

LET  A.TIME(NEW9)  -  TIME.V 

LET  TYPE (NEW9)  -  9 

LET  SERV. TIME (NEW9)  -  NORMAL. F (PI. SERV (9) ,  P2.SERV(9) f  7) 
LET  PRIOR (NEW9)  -  3 
CALL  PREEMPT  GIVEN  NEW9 

RESCHEDULE  THIS  I. ARRIVAL  IN  EXPONENTIAL. F (PI. ARR( 9) ,  7) 
SECONDS 
RETURN 
END 

EVENT  J. ARRIVAL  SAVING  THE  EVENT  NOTICE 

' 'ACQUIRE  TARGET  TASK 

DEFINE  NEWIO  AS  AN  INTEGER  VARIABLE 

CREATE  A  TASK  CALLED  NEWIO 

LET  NUM. ARRIVE  -  1 

LET  A.  TIME  (NEWIO)  -  TIME.V 

LET  TYPE (NEWIO)  -  10 

LET  SERV. TIME  (NEWIO)  « 

NORMAL. F (PI. SERV ( 10 ),  P2.SERV(10),  8) 

LET  PRIOR (NEWIO)  -  3 
CALL  PREEMPT  GIVEN  NEWIO 
RESCHEDULE  THIS  J. ARRIVAL  IN 

NORMAL. F ( PI. ARR( 10) #P2.ARR(10) ,8)  SECONDS 
RETURN 
END 

EVENT  K. ARRIVAL  SAVING  THE  EVENT  NOTICE 
•  'ACTIVATE  OFFENSIVE  SYSTEM  TAS** 

DEFINE  NEW11  AS  AN  INTEGER  VARIABLE 

CREATE  A  TASK  CALLED  NEW11 

LET  NUM. ARRIVE  -  1 

LET  A.TIME(NEWll)  -  TIME.V 

LET  TYPE(NEWll)  -  11 

LET  SERV. TIME (NEW11)  « 

NORMAL. F( PI. SERV ( 11) ,  P2.SERV(11),  9) 

LET  PRIOR (NEW11)  -  3 
CALL  PREEMPT  GIVEN  NEW11 
RETURN 
END 

EVENT  L. ARRIVAL  SAVING  THE  EVENT  NOTICE 
"COMMUNICATIONS  TASK 
DEFyNE  NEW12  AS  AN  INTEGER  VARIABLE 
CREATE  A  TASK  CALLED  NEW12 
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LET  NUM. ARRIVE  -  1 
LET  A.TIME(NEW12)  *  TIME.V 
LET  TYPE (NEW12)  -  12 
LET  SERV . TIME (NEW12 )  - 

NORMAL. F (PI * SERV (12) ,  P2.SERV(12) ,  S) 

LET  PRIOR (NEW12)  *  2 
CALL  PREEMPT  GIVEN  NEW12 
RETURN 
END 

EVENT  PEN.ACQ 

'  *  THIS  EVENT  CAUSES  A  TRANSITION  FROM  THE  PENETRATION 

• •  PHASE  TO  THE  TARGET  ACQUISITION  PHASE 

LET  P.IDLE  -  1  -  AV . BUSY 

LET  INT.l  -  INT . TOT/ ARR . TOT 

LET  INT. 4  *  INT. TOT/ (ARR.TOT-NCOUNT) 

LET  INT.?  -  POS. TOT/ (ARR.TOT-NCOUNT) 

LET  NCOUNT  -  0 
SCHEDULE  A  J. ARRIVAL  NOW 

RESET  THE  TOTALS  OF  BUSY,  INTER,  NUM. ARRIVE,  POSSIBLE. INT 


1.2*P1.ARR(1) 

0.75*P1.ARR(2) 

0.6*P1.ARR(3) 

0. 6*P1.ARR(4) 

0.6*P1.ARR(5) 

0.7*P1.ARR(6) 

Q.65*P1.ARR(9) 


LET  Pl.ARR(l) 

LET  P1.ARR(2) 

LET  P1.ARR(3) 

LET  P1.ARR(4) 

LET  P1.ARR(5) 

LET  P1.ARR(6) 

LET  P1.ARR(9) 

RETURN 
END 

EVENT  ACQ.WPN 

• '  THIS  EVENT  CAUSES  A  TRANSITION  FROM  THE  TARGET 

1  *  ACQUISTION  PHASE  TO  THE  WEAPON  DELIVERY  PHASE 

LET  A. IDLE  *  1  -  AV.BUSY 

LET  INT. 2  -  INT. TOT/ ARR. TOT 

LET  INT. 5  -  INT. TOT/ (ARR. TOT  -  NCOUNT) 

LET  INT. 8  -  POS. TOT/ (ARR. TOT  -  NCOUNT) 

LET  NCOUNT  -  0 

SCHEDULE  A  K. ARRIVAL  IN  10.0  SECONDS 

RESET  THE  TOTALS  OF  BUSY,  INTER,  NUM. ARRIVE,  POSSIBLE. INT 
LET  Pl.ARR(l)  -  0.4*P1.ARR(1) 

RETURN 

END 

ROUTINE  PREEMPT  GIVEN  NEW 
DEFINE  NEW  AS  AN  INTEGER  VARIABLE 
IF  BUSY (PILOT)  *  1 

LET  POSSIBLE, INT  *  1 
FOR  EACH  TASK  IN  WORK 
DO 

IF  PRIOR (NEW)  GT  PRIOR (TASK) 

REMOVE  THE  FIRST  TASK  FROM  WORK 
REMOVE  THIS  E.O.S  FROM  EV.S(I.E.O.S) 

LET  SERV. TIME (TASK)  * 

NORMAL.  F(  PI.  SERV  (TYPE  (TASK) ) , 
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P2 . SERV (TYPE (TASK) ) ,  10) 

FILE  THIS  TASK  IN  QUEUE 
FILE  THIS  NEW  IN  WORK 
LET  INTER  *  1 
LET  NUM. ARRIVE  »  1 
LET  N COUNT  «  NCOUNT  +1 

SCHEDULE  AN  E.O.S(NEW)  IN  SERV. TIME (NEW)  SECONDS 
ELSE 

FILE  THIS  NEW  IN  QUEUE 
ALWAYS 
LOOP 
ELSE 

FILE  THIS  NEW  IN  WORK 
LET  BUSY (PILOT)  -  1 

SCHEDULE  AN  E.O.S(NEW)  IN  SERV. TIME (NEW)  SECONDS 
ALWAYS 
RETURN 
END 

EVENT  E.O.S(TASK) 

DEFINE  TASK  AS  AN  INTEGER  VARIABLE 
IF  TYPE (TASK)  -  7 

SCHEDULE  AN  H. ARRIVAL  NOW 
ALWAYS 

REMOVE  THIS  TASK  FROM  WORK 
DESTROY  THIS  TASK  CALLED  TASK 
IF  QUEUE  IS  EMPTY 

LET  BUSY (PILOT)  -  0 

ELSE 

FOR  EACH  TASK  IN  QUEUE 
DO 

IF  TYPE (TASK)  -  12 

IF  TIME.V  -  A. TIME (TASK)  GT  50.0 
ADD  1  TO  PRIOR (TASK) 

ALWAYS 

ELSE 

IF  TIME.V  -  A. TIME (TASK)  GT  10.0 
ADD  1  TO  PRIOR (TASK) 

ALWAYS 

ALWAYS 

LOOP 

REMOVE  FIRST  TASK  FROM  QUEUE 
FILE  THIS  TASK  IN  WORK 

SCHEDULE  AN  E.O.S(TASK)  IN  SERV. TIME (TASK) 

SECONDS 

ALWAYS 

RETURN 

END 

EVENT  END. SIM 

• '  THIS  EVENT  ENDS  THE  SIMULATION  AND  COMPUTES  FINAL 

•'  STATISTICS 

LET  W. IDLE  »  1  -  AV . BUSY 

LET  INT.3  -  I NT. TOT/ ARR. TOT 
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LET  INT.6  -  INT . TOT/ ( ARR . TOT  -  NCOUNT) 

LET  INT. 9  *  POS. TOT/ (ARR. TOT  -  NCOUNT) 

LET  NCOUNT  =  0 

LET  Pl.ARR(l)  ^  2. 083*P1.ARR(1) 

LET  PI. ARR (2)  =  1. 333*P1. ARR(2) 

LET  P1.ARR(3)  =  1. 667*P1. ARR(3) 

LET  PI . ARR ( 4 )  =  1. 667*P1. ARR(4) 

LET  P1.ARR(5)  =  1. 667*P1.ARR(5) 

LET  PI. ARR ( 6)  *  1.429*P1.ARR(6) 

LET  PI. ARR (9)  =  1. 538*P1.ARR(9) 

LET  TIME . V  =  0 
LET  BUSY (PILOT)  *  0 

RESET  THE  TOTALS  OF  INTER,  NUM. ARRIVE,  BUS/,  POSSIBLE. INT 
FOR  EVERY  A. ARRIVAL  IN  EV.S(I.A.ARRIVAL) ,  DO 
REMOVE  THIS  A. ARRIVAL  FROM  EV.S (I. A. ARRIVAL) 

DESTROY  THIS  A. ARRIVAL 
LOOP 

FOR  EVERY  B. ARRIVAL  IN  EV. S (I. B. ARRIVAL) ,  DO 
REMOVE  THIS  B. ARRIVAL  FROM  EV. S( I. B. ARRIVAL) 

DESTROY  THIS  B. ARRIVAL 
LOOP 

FOR  EVERY  C, ARRIVAL  IN  EV.S (I. C. ARRIVAL) ,  DO 
REMOVE  THIS  C. ARRIVAL  FROM  EV. S( I. C. ARRIVAL) 

DESTROY  THIS  C. ARRIVAL 
LOOP 

FOR  EVERY  L. ARRIVAL  IN  EV. S( I. D. ARRIVAL) ,  DO 
REMOVE  THIS  D. ARRIVAL  FROM  EV . S ( I . D . ARRIVAL) 

DESTROY  THIS  D. ARRIVAL 
LOOP 

FOR  EVERY  E, ARRIVAL  IN  EV.S (I . E. ARRIVAL) ,  DO 
REMOVE  THIS  E. ARRIVAL  FROM  EV. S( I. E. ARRIVAL) 

DESTROY  THIS  E. ARRIVAL 
LOOP 

FOR  EVERY  F. ARRIVAL  IN  EV. S( I. F. ARRIVAL) ,  DO 
REMOVE  THIS  F. ARRIVAL  FROM  EV.S ( I. F. ARRIVAL) 

DESTROY  THIS  F. ARRIVAL 
LOOP 

FOR  EVERY  G. ARRIVAL  IN  EV.S (I. G. ARRIVAL) ,  DO 
REMOVE  THIS  G. ARRIVAL  FROM  EV . S ( I . G . ARRIVAL) 

DESTROY  THIS  G. ARRIVAL 
LOOP 

FOR  EVERY  H. ARRIVAL  IN  EV.S (I. H. ARRIVAL) ,  DO 
REMOVE  THIS  H. ARRIVAL  FROM  EV.S ( I. H. ARRIVAL) 

DESTROY  THIS  H. ARRIVAL 
LOOP 

FO  A.  EVERY  I .  ARRIVAL  IN  EV .  S  ( I .  I .  ARRIVAL)  ,  DO 
REMOVE  THIS  I. ARRIVAL  FROM  EV. S (I . I . ARRIVAL) 

DESTROY  THIS  I. ARRIVAL 
LOOP 

FOR  EVERY  J. ARRIVAL  IN  EV.S (I. J. ARRIVAL) ,  DO 
REMOVE  THIS  J. ARRIVAL  FROM  EV. S( I. J. ARRIVAL) 

DESTROY  THIS  J. ARRIVAL 


LOOP 

FOR  EVERY  K. ARRIVAL  IN  EV.S (I .K. ARRIVAL) ,  DO 
REMOVE  THIS  K. ARRIVAL  FROM  EV. S( I. K. ARRIVAL) 

DESTROY  THIS  K. ARRIVAL 
LOOP 

FOR  EVERY  L. ARRIVAL  IN  EV.S (I . L. ARRIVAL) ,  DO 
REMOVE  THIS  L. ARRIVAL  FROM  EV. S( I. L. ARRIVAL) 

DESTROY  THIS  L. ARRIVAL 
LOOP 

FOR  EVERY  E.O.S  IN  EV.S(I.E.O.S) ,  DO 
REMOVE  THIS  E.O.S  FROM  EV.S (I . E.O.S) 

DESTROY  THIS  E.O.S 
LOOP 

FOR  EVERY  TASK  IN  WORK,  DO 
REMOVE  THIS  TASK  FROM  WORK 
DESTROY  THIS  TASK 
LOOP 

FOR  EVERY  TASK  IN  QUEUE,  DO 
REMOVE  THIS  TASK  FROM  QUEUE 
DESTROY  THIS  TASK 
LOOP 
RETURN 
END 

EVENT  DATA • RESET 

RESET  THE  TOTALS  OF  BUSY,  INTER,  NUM. ARRIVE,  POSSIBLE. INT 

LET  NCOUNT  *  0 

RETURN 

END 

ROUTINE  SNAP . R 

FOR  EACH  TASK  IN  QUEUE,  LIST  ATTRIBUTES  OF  TASK 
FOR  EACH  TASK  IN  WORK,  LIST  ATTRIBUTES  OF  TASK 
LIST  ATTRIBUTES  OF  EACH  A. ARRIVAL  IN  EV.S (I. A. ARRIVAL) 
LIST  ATTRIBUTES  OF  EACH  B. ARRIVAL  IN  EV.S (I. B. ARRIVAL) 
LIST  ATTRIBUTES  OF  EACH  C. ARRIVAL  IN  EV.S (I . C. ARRIVAL) 
LIST  ATTRIBUTES  OF  EACH  D. ARRIVAL  IN  EV.S (I . D. ARRIVAL) 
LIST  ATTRIBUTES  OF  EACH  E. ARRIVAL  IN  EV.S (I. E. ARRIVAL) 
LIST  ATTRIBUTES  OF  EACH  F. ARRIVAL  IN  EV . S ( I . F . ARRIVAL) 
LIST  ATTRIBUTES  OF  EACH  G. ARRIVAL  IN  EV.S (I. G. ARRIVAL) 
LIST  ATTRIBUTES  OF  EACH  H. ARRIVAL  IN  EV.S (I. H. ARRIVAL) 
LIST  ATTRIBUTES  OF  EACH  I. ARRIVAL  IN  EV.S (I. I. ARRIVAL) 
LIST  ATTRIBUTES  OF  EACH  J. ARRIVAL  IN  EV.S (I .J. ARRIVAL) 
LIST  ATTRIBUTES  OF  EACH  K. ARRIVAL  IN  EV.S (I . K. ARRIVAL) 
LIST  ATTRIBUTES  OF  EACH  L. ARRIVAL  IN  EV.S (I. L. ARRIVAL) 
LIST  ATTRIBUTES  OF  EACH  E.O.S  IN  EV.S(I. E.O.S) 

LIST  TIME . V ,  N . WORK ,  N. QUEUE,  BUSY (PILOT) 

SKIP  2  LINES 

RETURN 

END 
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PARALLEL  PROCESSING  MODEL: 


PREAMBLE  LAST  COLUMN  IS  72  ' ' 

"PROGRAM  MRMABL  -  COMPLETE  MISSION,  25  REPS 
EVENT  NOTICES  INCLUDE  END. SIM,  DATA. RESET, 

A. ARRIVAL,  B. ARRIVAL,  C. ARRIVAL,  D. ARRIVAL, 

E. ARRIVAL,  F. ARRIVAL,  G. ARRIVAL,  H. ARRIVAL, 

I. ARRIVAL,  J. ARRIVAL,  K. ARRIVAL,  L. ARRIVAL, 

PEN.ACQ,  ACQ.WPN 
EVERY  E.O.S  HAS  A  JOB 

THE  SYSTEM  OWNS  A  QUEUE,  A  ’.ORK,  A  TEMPI,  A  TEMP2,  AND 
A  TEMP 3 

PERMANENT  ENTITIES  INCLUDE  PILOT 
TEMPORARY  ENTITIES 

EVERY  TASK  HAS  AN  A. TIME,  A  TYPE,  A  PRIOR, 

A  SERV . TIME ,  A  COMP. TIME,  A  REM. TIME,  A  MODAL, 

A  CODE,  A  STAGE,  AN  EVPOINTER  AND  MAY  BELONG  TO  THE 
QUEUE,  THE  WORK,  THE  TEMPI,  THE  TEMP2,  AND  THE  TEMP3 
DEFINE  SECONDS  TO  MEAN  UNITu, 

DEFINE  SIM. TIME,  A. TIME, 

P. IDLE,  A. IDLE,  W.IDLE,  SIM.l,  SIM. 2,  SIM. 3, 

UP. TOTAL,  DEG. TOTAL,  SERV. TIME 
AS  REAL  VARIABLES 

DEFINE  BUSY,  TYPE,  PRIOR,  N,  NUM. ARRIVE,  JOB,  MODAL, 

CODE,  STAGE,  SIMUL,  EVPOINTER 
AS  INTEGER  VARIABLES 

DEFINE  Pl.ARR,  P2.ARR,  PI. SERV,  P2.SERV  AS  1-DIM  VARIABLES 

DEFINE  MRM,  TEST  AS  INTEGER,  2 -DIM  VARIABLES 

DEFINE  QUEUE  AS  A  SET  RANKED  BY  PRIOR 

DEFINE  WORK  AS  A  SET  RANKED  BY  LOW  COMP. TIME 

DEFINE  TEMPI  AS  A  SET  RANKED  BY  REM. TIME 

DEFINE  TEMP2  AS  A  SET  RANKED  BY  REM. TIME 

DEFINE  PREEMPT  AS  A  ROUTINE  GIVEN  2  ARGUMENTS 

DEFINE  DEGRADE  AS  A  ROUTINE  GIVEN  1  ARGUMENT 

DEFINE  UPGRADE  AS  A  ROUTINE  GIVEN  1  ARGUMENT 

DEFINE  CHANGE  AS  A  ROUTINE  GIVEN  1  ARGUMENTS 

DEFINE  ADJUST  AS  A  ROUTINE  GIVEN  2  ARGUMENTS 

DEFINE  RECOVER  AS  A  ROUTINE 

DEFINE  CORRECT  AS  A  ROUTINE 

TALLY  SIM. TOT  AS  THE  SUM  OF  SIMUL 

TALLY  ARR.TOT  AS  THE  SUM  OF  NUM. ARRIVE 

ACCUMULATE  AV.BUSY  AS  THE  MEAN  OF  BUSY 

' '  THE  FOLLOWING  STATEMENTS  ARE  USED  TO  COLLECT  FINAL 

' '  STATISTICS 

TALLY  PEN. MEAN  AS  THE  MEAN,  PEN. VAR  AS  THE  VARIANCE  OF 
P. IDLE 

TALLY  ACQ . MEAN  AS  THE  MEAN,  ACQ.VAR  AS  THE  VARIANCE  OF 
A. IDLE 

TALLY  WPN . MEAN  AS  THE  MEAN,  WPN.VAR  AS  THE  VARIANCE  OF 
W.IDLE 

TALLY  PEN1.SIM  AS  THE  MEAN,  PEN2.SIM  AS  THE  VARIANCE  OF 


SIM.  1 

TALLY  ACQ1.SIM  AS  THE  MEAN,  ACQ2.SIM  AS  THE  VARIANCE  OF 
SIM.  2 

TALLY  WPN1.SIM  AS  THE  MEAN,  WPN2.SIM  AS  THE  VARIANCE  OF 
SIM.  3 

END 

MAIN 

DEFINE  II,  J,  REPS  AS  INTEGER  VARIABLES 
READ  SIM. TIME,  RESET. TIME 
START  NEW  CARD 
READ  N  REPS 

PRINT  4  LINES  WITH  N,  REPS  THUS 

THIS  EXPERIMENT  USES  A  MULTIPLE  RESOURCE  MODEL 

TO  DESCRIBE  A  PILOT  IN  AN  AIR-TO-GROUND  MISSION. 

**  DIFFERENT  TASKS  ARE  USED  TO  LOAD  THE  PILOT. 

***  REPLICATIONS  OF  EACH  MISSION  ARE  ACCOMPLISHED. 

SKIP  2  LINES 

RESERVE  PI. ARR(*)  AS  N,  P1.SERV(*)  AS  N, 

P2.ARR(*)  AS  N,  P2.SERV(*)  AS  N 
RESERVE  MRM(*,*),  TEST ( * , * )  AS  2  BY  3 
PRINT  2  LINES  THUS 

TASK  NUMBER  ARRIVAL  RATE  SERVICE  RATE 

PARAM  1  PARAM  2  PARAM  1  PARAM  2 
FOR  II  *  1  TO  N,  DO 
START  NEW  CARD 

READ  Pl.ARR(II),  P2.ARR(II),  Pl.SERV(II) ,  P2.SERV(II) 
PRINT  1  LINE  WITH  II,  Pl.ARR(II) ,  P2.ARR(II), 
Pl.SERV(II) ,  P2 .SERV(II)  THUS 

***  *,***  *,***  it,***  *,*** 

LOOP 

SKIP  2  LINES 

PRINT  2  LINES  WITH  SIM. TIME,  RESET. TIME  THUS 
TOTAL  SIMULATION  TIME  *  ******  SECONDS 

DATA  RESET  TIME  =  ***♦.*  SECONDS 

SKIP  2  LINES 
LET  N. PILOT  =  1 
CREATE  EVERY  PILOT 
LET  PILOT  =  1 
FOR  J  -  1  TO  REPS,  DO 

SCHEDULE  AN  A. ARRIVAL  NOW 
SCHEDULE  A  B. ARRIVAL  NOW 
SCHEDULE  A  C. ARRIVAL  NOW 
SCHEDULE  A  D. ARRIVAL  NOW 
SCHEDULE  AN  E. ARRIVAL  NOW 
SCHEDULE  AN  F. ARRIVAL  NOW 
SCHEDULE  A  G. ARRIVAL  IN  300.0  SECONDS 
SCHEDULE  AN  I. ARRIVAL  IN  EXPONENTIAL. F( PI. ARR (9) , 10) 
SECONDS 

SCHEDULE  A  PEN.ACQ  IN  500  SECONDS 
SCHEDULE  A  ACQ.WPN  IN  545  SECONDS 
SCHEDULE  AN  L. ARRIVAL  IN  250.0  SECONDS 
SCHEDULE  A  DATA. RESET  IN  RESET. TIME  SECONDS 
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SCHEDULE  AN  END. SIM  IN  SIM. TIME  SECONDS 
START  SIMULATION 
LOOP 

PRINT  2  LINES  THUS 

IDLE  TIME  RATIO  SUMMARY 
PHASE  MEAN 

PRINT  3  LINES  WITH  PEN. MEAN, 

ACQ.MEAN, 

WPN.MEAN, 

PENETRATION  * . ***** 

TARGET  ACQUISITION  *.***** 

WEAPON  DELIVERY  *.***** 

SKIP  2  LINES 
PRINT  2  LINES  THUS 

SIMULTANEOUS  TASK 
PHASE  MEAN 

PRINT  3  LINES  WITH  PEN1.SIM, 

ACQ1.SIM, 

WPN1.SIM, 

PENETRATION  * . ***** 

TARGET  ACQUISITION  *.***** 

WEAPON  DELIVERY  *.***** 

CALL  SNAP . R 
STOP 
END 

EVENT  A. ARRIVAL  SAVING  THE  EVENT  NOTICE 

DEFINE  NEW1 ,  KEY  AS  INTEGER  VARIABLES 

CREATE  A  TASK  CALLED  NEW1 

LET  NUM. ARRIVE  «  1 

LET  A.TIME(NEWl)  *  TIME.V 

LET  TYPE (NEW1)  =  1 

LET  SERV . TIME ( NEW1 )  =  EXPONENTIAL. F( PI. SERV(l) ,  1) 

LET  PRIOR (NEW1)  =  3 

LET  MODAL (NEW1)  =  1 

LET  CODE (NEW1)  =  2 

LET  STAGE (NEW1)  =  2 

LET  KEY  =  0 

IF  N . WORK  =  0 

LET  DEG. TOTAL  =  0 
CALL  ADJUST  GIVEN  NEW1,  KEY 
ELSE 

CALL  DEGRADE  GIVEN  NEW1 
IF  DEG. TOTAL  LT  1.0 

CALL  ADJUST  GIVEN  NEW1,  KEY 
ELSE 

IF  PRIOR (NEW1)  GT  1 

CALL  PREEMPT  GIVEN  NEW1,  KEY 
ELSE 

FILE  NEW1  IN  QUEUE 
ALWAYS 
ALWAYS 
ALWAYS 


STANDARD  DEVIATION 
SQRT . F ( PEN . VAR/REPS ) , 

SQRT . F ( ACQ . VAR/REPS ) , 
SQRT. F(WPN. VAR/REPS)  THUS 
* .  ***11* 

*,***** 

* . ***** 


RATE  SUMMARY 

STANDARD  DEVIATION 
SQRT . F ( PEN2 . SIM/REPS ) , 

SQRT . F (ACQ2 . SIM/REPS ) , 
SQRT. F(WPN2. SIM/REPS)  THUS 
* . ***** 

* . ***** 
*,***** 


265 


CALL  STATUS 

RESCHEDULE  THIS  A. ARRIVAL  IN 

NORMAL. F(P1. ARR(l) ,  P2.ARR(1),  1)  SECONDS 
RETURN 
END 

EVENT  B. ARRIVAL  SAVING  THE  EVENT  NOTICE 

DEFINE  NEW2,  KEY  AS  INTEGER  VARIABLES 

CREATE  A  TASK  CALLED  NEW2 

LET  NUM. ARRIVE  *  1 

LET  A. TIME (NEW2)  *  TIME.V 

LET  TYPE(NEW2)  *  2 

LET  SERV . TIME (NEW2 )  ■  NORMAL. F( PI. SERV (2 ) ,  P2 .SERV(2) , 2) 

LET  PRIOR (NEW2)  =2 

LET  MODAL (NEW2)  =  1 

LET  CODE (NEW2)  *  1 

LET  STAGE (NEW2)  =  1 

LET  KEY  =  0 

IF  N . WORK  ■  0 

LET  DEG. TOTAL  =  0. 

CALL  ADJUST  GIVEN  NEW2,  KEY 
ELSE 

CALL  DEGRADE  GIVEN  NEW2 
IF  DEG. TOTAL  LT  1.0 

CALL  ADJUST  GIVEN  NEW2,  KEY 
ELSE 

IF  PRIOR (NEW2)  GT  1 

CALL  PREEMPT  GIVEN  NEW2 ,  KEY 
ELSE 

FILE  NEW2  IN  QUEUE 
ALWAYS 
ALWAYS 
ALWAYS 
CALL  STATUS 

RESCHEDULE  THIS  B. ARRIVAL  IN 

NORMAL.F(Pl.ARR(2) ,  P2.ARR(2),  2)  SECONDS 
RETURN 
END 

EVENT  C. ARRIVAL  SAVING  THE  EVENT  NOTICE 

DEFINE  NEW3 ,  KEY  AS  INTEGER  VARIABLES 

CREATE  A  TASK  CALLED  NEW3 

LET  NUM. ARRIVE  =  1 

LET  A.TIME(NEW3)  =  TIME.V 

LET  TYPE (NEW3)  *  3 

LET  SERV. TIME (NEW3)  =  NORMAL. F (PI. SERV (3 ) ,  P2 .SERV(3) , 3) 

LET  PRIOR (NEW3)  =  1 

LET  MODAL (NEW3)  -  1 

LET  CODE (NEW3)  =  2 

LET  STAGE (NEW3)  =  1 

LET  KEY  =  0 

IF  N . WORK  =  0 

LET  DEG. TOTAL  -  0. 

CALL  ADJUST  GIVEN  NEW3 ,  KEY 
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ELSE 

CALL  DEGRADE  GIVEN  NEW3 
IF  DEG. TOTAL  LT  1.0 

CALL  ADJUST  GIVEN  NEW3,  KEY 
ELSE 

IF  PRIOR (NEW3)  GT  1 

CALL  PREEMPT  GIVEN  NEW3 ,  KEY 
ELSE 

FILE  NEW 3  IN  QUEUE 
ALWAYS 
ALWAYS 
ALWAYS 
CALL  STATUS 

RESCHEDULE  THIS  C. ARRIVAL  IN 

NORMAL. F(P1.ARR(3) ,  P2.ARR(3),  3)  SECONDS 
RETURN 
END 

EVENT  D. ARRIVAL  SAVING  THE  EVENT  NOTICE 

DEFINE  NEW4,  KEY  AS  INTEGER  VARIABLES 

CREATE  A  TASK  CALLED  NEW4 

LET  NUM. ARRIVE  =  1 

LET  A.TIME(NEW4)  -  TIME.V 

LET  TYPE(NEW4)  »  4 

LET  SERV . TIME (NEW4 )  »  N0RMAL.F(P1.SERV(4) ,  P2.SERV(4),  4) 

LET  PRIOR (NEW4)  -  1 

LET  MODAL (NEW4)  -  1 

LET  C0DE(NEW4)  =•  1 

LET  STAGE (NEW4)  ■  1 

LET  KEY  *  0 

IF  N.WORK  »  0 

LET  DEG. TOTAL  -  0 
CALL  ADJUST  GIVEN  NEW4,  KEY 
ELSE 

CALL  DEGRADE  GIVEN  NEW4 
IF  DEG. TOTAL  LT  1.0 

CALL  ADJUST  GIVEN  NEW4,  KEY 
ELSE 

IF  PRIOR (NEW4)  GT  1 

CALL  PREEMPT  GIVEN  NEW4,  KEY 
ELSE 

FILE  NEW4  IN  QUEUE 
ALWAYS 
ALWAYS 
ALWAYS 
CALL  STATUS 

RESCHEDULE  THIS  D. ARRIVAL  IN 

NORMAL. F(P1. ARR(4) ,  P2.ARR(4),  4)  SECONDS 
RETURN 
END 

EVENT  E. ARRIVAL  SAVING  THE  EVENT  NOTICE 
DEFINE  NEW5,  KEY  AS  INTEGER  VARIABLES 
CREATE  A  TASK  CALLED  NEW5 
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LET  NUM. ARRIVE  «  1 

LET  A. TIME (NEW5)  ■  TIME.V 

LET  TYPE(NEW5)  -  5 

LET  SERV . TIME (NEWS)  «  NORMAL. F( PI. SERV ( 5) ,  P2  .SERV(5) , 5) 

LET  PRIOR (NEW5)  *  2 

LET  MODAL (NEW5)  -  1 

LET  CODE (NEWS)  -  1 

LET  STAGE (NEW5)  =  1 

LET  KEY  *  0 

IF  N.WORK  -  0 

LET  DEG. TOTAL  *  0. 

CALL  ADJUST  GIVEN  NEWS,  KEY 
ELSE 

CALL  DEGRADE  GIVEN  NEW5 
IF  DEG. TOTAL  LT  1.0 

CALL  ADJUST  GIVEN  NEW5,  KEY 
ELSE 

IF  PRIOR (NEW5)  GT  1 

CALL  PREEMPT  GIVEN  NEWS,  KEY 
ELSE 

FILE  NEW 5  IN  QUEUE 
ALWAYS 
ALWAYS 
ALWAYS 
CALL  STATUS 

RESCHEDULE  THIS  E. ARRIVAL  IN 

N0RMAL.F(P1.ARR(5) ,  P2.ARR(5),  5)  SECONDS 
RETURN 
END 

EVENT  F. ARRIVAL  SAVING  THE  EVENT  NOTICE 

DEFINE  NEWS,  KEY  AS  INTEGER  VARIABLES 

CREATE  A  TASK  CALLED  NEW6 

LET  NUM. ARRIVE  -  1 

LET  A.TIME(NEW6)  -  TIME.V 

LET  TYPE(NEW6)  -  6 

LET  SERV. TIME (NEW6)  -  NORMAL. F( PI. SERV ( 6) ,  P2 .SERV(6) , 6) 

LET  PRIOR (NEW6)  -  3 

LET  MODAL (NEW6)  -  1 

LET  CODE (NEW6)  -  2 

LET  STAGE (NEW6)  -  2 

LET  KEY  -  0 

IF  N.WORK  -  0 

LET  DEG. TOTAL  -  0. 

CALL  ADJUST  GIVEN  NEW6,  KEY 
ELSE 

CALL  DEGRADE  GIVEN  NEW 6 
IF  DEG. TOTAL  LT  1.0 

CALL  ADJUST  GIVEN  NEW6,  KEY 
ELSE 

IF  PRIOR (NEW6)  GT  1 

CALL  PREEMPT  GIVEN  NEW6,  KEY 
ELSE 
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FILE  NEW6  IN  QUEUE 
ALWAYS 
ALWAYS 
ALWAYS 
CALL  STATUS 

RESCHEDULE  THIS  F. ARRIVAL  IN  NORMAL. F( PI. ARR  %I  SECONDS 

RETURN 

END 

EVENT  G. ARRIVAL  SAVING  THE  EVENT  NOTICE 

DEFINE  NEW7 ,  KEY  AS  INTEGER  VARIABLES 

CREATE  A  TASK  CALLED  NEW7 

LET  NUM. ARRIVE  -  1 

LET  A.TIME(NEW7)  -  TIME.V 

LET  TYPE(NEW7)  -  7 

LET  SERV. TIME (NEW7)  -  NORMAL.F(Pl.SERV(7) ,  P2.SERV(7)  ,9) 

LET  PRIOR (NEW7)  -  3 

LET  MODAL (NEW7)  -  1 

LET  CODE(NEW7)  -  2 

LET  STAGE (NEW7)  -  1 

LET  KEY  -  0 

IF  N.WORK  -  0 

LET  DEG. TOTAL  -  0 
CALL  ADJUST  GIVEN  NEW7,  KEY 
ELSE 

CALL  DEGRADE  GIVEN  NEW7 
IF  DEG. TOTAL  LT  1.0 

CALL  ADJUST  GIVEN  NEW7 ,  KEY 
ELSE 

IF  PRIOR (NEW7)  GT  1 

CALL  PREEMPT  GIVEN  NEW7 ,  KEY 
ELSE 

FILE  NEW7  IN  QUEUE 
ALWAYS 
ALWAYS 
ALWAYS 
CALL  STATUS 
RETURN 
END 

EVENT  H. ARRIVAL  SAVING  THE  EVENT  NOTICE 

DEFINE  NEW8,  KEY  AS  INTEGER  VARIABLES 

CREATE  A  TASK  CALLED  NEW8 

LET  NUM. ARRIVE  -  1 

LET  A. TIME (NEWS)  -  TIME.V 

LET  TYPE(NEW8)  -  8 

LET  SERV. TIME (NEW8)  -  NORMAL. F (PI. SERV (8) ,  P2 .SERV(8) , 9) 

LET  PRIOR (NEW8)  -  3 

LET  MODAL (NEW8)  -  1 

LET  CODE(NEW8)  -  2 

LET  STAGE (NEW8)  -  2 

LET  KEY  -  0 

IF  N.WORK  *  0 

LET  DEG. TOTAL  -  0. 


CALL  ADJUST  GIVEN  NEW8,  KEY 
ELSE 

CALL  DEGRADE  GIVEN  NEW8 
IF  DEG. TOTAL  LT  1.0 

CALL  ADJUST  GIVEN  NEW8,  KEY 
ELSE 

IF  PRIOR (NEW8)  GT  1 

CALL  PREEMPT  GIVEN  NEW8 ,  KEY 
ELSE 

FILE  NEW8  IN  QUEUE 
ALWAYS 
ALWAYS 
ALWAYS 
CALL  STATUS 
RETURN 
END 

EVENT  I. ARRIVAL  SAVING  THE  EVENT  NOTICE 

DEFINE  NEWS,  KEY  AS  INTEGER  VARIABLES 

CREATE  A  TASK  CALLED  NEW9 

LET  NUM. ARRIVE  *»  1 

LET  A.TIME(NEW9)  -  TIME.V 

LET  TYPE(NEW9)  -  9 

LET  SERV . TIME (NEW9 )  -  NORMAL. F(P1.SERV(9) ,  P2 .SERV(9) , 7) 

LET  PRIOR (NEW9)  -  3 

LET  MODAL (NEW9)  -  1 

LET  CODE(NEW9)  »  2 

LET  STAGE (NEW9)  ■  2 

LET  KEY  -  0 

IF  N.WORK  -  0 

LET  DEG. TOTAL  «  0. 

CALL  ADJUST  GIVEN  NEW9,  KEY 
ELSE 

CALL  DEGRADE  GIVEN  NEW9 
IF  DEG. TOTAL  LT  1.0 

CALL  ADJUST  GIVEN  NEW9,  KEY 
ELSE 

IF  PRIOR (NEW9)  GT  1 

CALL  PREEMPT  GIVEN  NEW9,  KEY 
ELSE 

FILE  NEW9  IN  QUEUE 
ALWAYS 
ALWAYS 
ALWAYS 
CALL  STATUS 

RESCHEDULE  THIS  I. ARRIVAL  IN 

EXPONENTIAL. F( PI. ARR (9 ) ,  7)  SECONDS 
RETURN 
END 

EVENT  J. ARRIVAL  SAVING  THE  EVENT  NOTICE 
DEFINE  NEW10,  KEY  AS  INTEGER  VARIABLES 
CREATE  A  TASK  CALLED  NEW10 
LET  NUM. ARRIVE  -  1 


LET  A. TIME (NEW10)  -  TIME.V 
LET  TYPE(NEWIO)  *  10 
LET  SERV . TIME ( NEW10 )  - 

NORMAL. F (PI. SERV( 10) ,  P2.SERV(10),  8) 

LET  PRIOR (NEW10)  *  3 
LET  MODAL(NEWIO)  *  1 
LET  CODE(NEWIO)  -  2 
LET  STAGE (NEW10)  -  1 
LET  KEY  -  0 
IF  N.WORK  -  0 

LET  DEG. TOTAL  -  0 
CALL  ADJUST  GIVEN  NEW10,  KEY 
ELSE 

CALL  DEGRADE  GIVEN  NEW 10 
IF  DEG. TOTAL  LT  1.0 

CALL  ADJUST  GIVEN  NEW10,  KEY 
ELSE 

IF  PRIOR (NEW10)  GT  1 

CALL  PREEMPT  GIVEN  NEW10,  KEY 
ELSE 

FILE  NEW 10  IN  QUEUE 
ALWAYS 
ALWAYS 
ALWAYS 
CALL  STATUS 

RESCHEDULE  THIS  J. ARRIVAL  IN 

NORMAL. F( PI. AJW(  10)  ,  P2.ARR(10),  8)  SECONDS 
RETURN 
END 

EVENT  K. ARRIVAL  SAVING  THE  EVENT  NOTICE 

DEFINE  NEW11,  KEY  AS  INTEGER  VARIABLES 

CREATE  A  TASK  CALLED  NEW11 

LET  NUN. ARRIVE  «  1 

LET  A.TIME(NEWll)  -  TIME.V 

LET  TYPE(NEWll)  -  11 

LET  SERV. TIME (NEW11)  - 

NORMAL. F( PI. SERV (11) ,  P2.SERV(11) , 9) 

LET  PRIOR (NEW11)  -  3 
LET  MODAL (NEW11)  -  1 
LET  CODE (NEW11)  -  2 
LET  STAGE (NEW11)  -  2 
LET  KEY  -  0 
IF  N.WORK  -  0 

LET  DEG. TOTAL  -  0. 

CALL  ADJUST  GIVEN  NEWU,  KEY 
ELSE 

CALL  DEGRADE  GIVEN  NEW11 
IF  DEG. TOTAL  LT  1.0 

CALL  ADJUST  GIVEN  NEW11,  KEY 
ELSE 

IF  PRIOR (NEW11)  GT  1 

CALL  PREEMPT  GIVEN  NEW11,  KEY 
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ELSE 

FILE  NEW 11  IN  QUEUE 
ALWAYS 
ALWAYS 
ALWAYS 
CALL  STATUS 
RETURN 
END 

EVENT  L. ARRIVAL  SAVING  THE  EVENT  NOTICE 

DEFINE  NEW 12 ,  KEY  AS  INTEGER  VARIABLES 

CREATE  A  TASK  CALLED  NEW 12 

LET  NUM. ARRIVE  «  1 

LET  A . TIME ( NEW12 )  -  TIME.V 

LET  TYPE(NEW12)  »  12 

LET  SERV * TIME (NEW12 )  » 

NORMAL. F( PI. SERV( 12 ) ,  P2 .SERV(12) , 9) 

LET  PRIOR (NEW12)  -  2 
LET  MODAL (NEW12)  -  2 
LET  CODE(NEW12)  -  1 
LET  STAGE (NEW12)  -  1 
LET  KEY  »  0 
IF  N. WORK  -  0 

LET  DEG. TOTAL  -  0. 

CALL  ADJUST  GIVEN  NEW 12 ,  KEY 
ELSE 

CALL  DEGRADE  GIVEN  NEW12 
IF  DEG. TOTAL  LT  1.0 

CALL  ADJUST  GIVEN  NEW12,  KEY 
ELSE 

IF  PRIOR (NEW12)  GT  1 

CALL  PREEMPT  GIVEN  NEW12,  KEY 
ELSE 

FILE  NEW12  IN  QUEUE 
ALWAYS 
ALWAYS 
ALWAYS 
CALL  STATUS 
RETURN 
END 

EVENT  PEN. ACQ 

**  THIS  EVENT  CAUSES  A  TRANSITION  FROM  THE  PENETRATION 

4 '  PHASE  TO  THE  TARGET  ACQUISITION  PHASE 

LET  P. IDLE  -  1  -  AV.BUSY 

LET  SIM. 1  -  SIM. TOT/ ARR. TOT 

SCHEDULE  A  J. ARRIVAL  NOW 

RESET  THE  TOTALS  OF  BUSY,  NUM. ARRIVE,  SIMUL 
LET  Pl.ARR(l)  -  1.2*P1.ARR(1) 

LET  P1.ARR(2)  «  0.75*P1.ARR(2) 

LET  P1.ARR(3)  «  0.6*P1. ARR(3) 

LET  P1.ARR(4)  «  0.6*P1.ARR(4) 

LET  P1.ARR(5)  »  0.6*P1. ARR(5) 

LET  P1.ARR{6)  -  0.7*P1.ARR(6) 


LET  P1.ARR(9)  -  0. 65*P1.ARR(9) 

RETURN 

END 

EVENT  ACQ.WPN 

»  »  THIS  EVENT  CAUSES  A  TRANSITION  FROM  THE  TARGET 

• '  ACQUISITION  PHASE  TO  THE  WEAPON  DELIVERY  PHASE 

LET  A. IDLE  -  1  -  AV.BUSY 

LET  SIM. 2  »  SIM. TOT/ ARR. TOT 

SCHEDULE  A  K. ARRIVAL  IN  10.0  SECONDS 

RESET  THE  TOTALS  OF  BUSY,  NUM. ARRIVE,  SIMUL 

LET  Pl.ARR(l)  -  0.4*P1.ARR(1) 

RETURN 

END 

ROUTINE  STATUS 
IF  N.WORK  GE  1 
LET  BUSY  »  1 
ELSE 

LET  BUSY  »  0 
ALWAYS 
RETURN 
END 

ROUTINE  PREEMPT  GIVEN  NEW,  KEY 

*  *  THIS  ROUTINE  DETERMINES  IF  A  TASK  IN  WORK  SHOULD  BE 
• '  PREEMPTED  AND,  IF  SO,  INITIATES  THAT  ACTION 
DEFINE  NEW,  KEY  AS  INTEGER  VARIABLES 
FOR  EACH  TASK  IN  WORX 
DO 

LET  REM. TIME (TASK)  -  COMP. TIME (TASK)  -  TIME.V 
IF  PRIOR (TASK)  -  1 

REMOVE  THIS  TASK  FROM  WORK 
FILE  THIS  TASK  IN  TEMPI 
ELSE 

IF  PRIOR (TASK)  ~  2 

REMOVE  THIS  TASK  FROM  WORK 
FILE  THIS  TASK  IN  TEMP2 
ALWAYS 
ALWAYS 
LOOP 

IF  N. TEMPI  GT  0 

FOR  EACH  TASK  IN  TEMPI, 

UNTIL  DEG. TOTAL  LT  1.0 
DO 

REMOVE  THE  FIRST  TASK  FROM  TEMPI 
CALL  UPGRADE  GIVEN  TASK 
CALL  CHANGE  GIVEN  TASK 
CALL  CORRECT 
LET  NUM. ARRIVE  -  1 
FILE  THIS  TASK  IN  TEMPI 
CALL  DEGRADE  GIVEN  NEW 
LOOP 

IF  DEG. TOTAL  LT  1.0 
CALL  RECOVER 
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CALL  ADJUST  GIVEN  NEW,  KEY 
ALWAYS 
ALWAYS 

IF  M. WORK (NEW)  *  0 
IF  PRIOR (NEW)  LT  3 
IF  KEY  *  0 

FILE  NEW  IN  QUEUE 
CALL  RECOVER 
ALWAYS 
ELSE 

FOR  EACH  TASK  IN  TEMP2, 

UNTIL  DEG. TOTAL  LT  1,0 
DO 

REMOVE  FIRST  TASK  FROM  TEMP2 
CALL  UPGRADE  GIVEN  TASK 
CALL  CHANGE  GIVEN  TASK 
CALL  CORRECT 
FILE  THIS  TASK  IN  TEMP3 
LET  NUM. ARRIVE  -  1 
CALL  DEGRADE  GIVEN  NEW 
LOOP 

IF  DEG. TOTAL  LT  1.0 
CALL  RECOVER 

CALL  ADJUST  GIVEN  NEW,  KEY 
ELSE 

IF  KEY  -  0 

FILE  NEW  IN  QUEUE 
CALL  RECOVER 
ALWAYS 
ALWAYS 
ALWAYS 
ELSE 

CALL  RECOVER 
ALWAYS 

CALL  RECOVER 

RETURN 

EM} 

ROUTINE  ADJUST  GIVEN  NEW,  KEY 

» *  THIS  ROUTINE  RESCHEDULES  TASK  COMPLETION  TIMES  IN  WORK 

* »  AND  UPDATES  STATUS  OF  RESOURCES  BEING  USED 

1 ’  WHEN  AN  ADDITIONAL  TASK  IS  ASSIGNED  TO  WORK 

DEFINE  NEW,  KEY  AS  INTEGER  VARIABLES 

FOR  EACH  TASK  IN  WORK 

DO 

LET  REM. TIME (TASK)  -  COMP. TIME (TASK)  -  TIME.V 
LET  SER7. TIME (TASK)  »  REM. TIME (TASK)  +  DEG. TOTAL  * 

REM. TIME (TASK) 

REMOVE  THIS  EVPOINTER (TASK)  FROM  EV.S(I.E.O.S) 

SCHEDULE  AN  E.O.S(TASK)  IN  SERV. TIME (TASK)  SECONDS 
LET  EVPOINTER (TASK)  «  E.O.S 

LET  COMP. TIME (TASK)  -  TIME.V  +  SERV. TIME (TASK) 

LOOP 


LET  SERV. TIME (NEW)  -  SERV. TIME (NEW)  + 

DEG. TOTAL  *  SERV. TIME (NEW) 

LET  COMP. TIME (NEW)  *  TIME.V  +  SERV. TIME (NEW) 

IF  KEY  *  1 

REMOVE  THIS  NEW  FROM  QUEUE 
ALWAYS 

FILE  NEW  IN  WORK 

SCHEDULE  AN  E.O.S(NEW)  IN  SERV. TIME (NEW)  SECONDS 
LET  EVPO INTER (NEW)  -  E.O.S 
IF  MODAL (NEW)  *  1 
ADD  i  TO  MRM(2, 1) 

ELSE 

ADD  1  TO  MRM(1,1) 

ALWAYS 

IF  CODE (NEW)  -  1 
ADD  1  TO  MRM(2,2) 

ELSE 

ADD  I  TO  MRM(1,2) 

ALWAYS 

IF  STAGE (NEW)  «  1 
ADD  1  TO  KRM(2, 3) 

ELSE 

ADD  1  TO  KRM(1, 3) 

ALWAYS 

IF  N.WORK  GT  1 
LET  SIMUL  »  1 
ALWAYS 
RETURN 
END 

ROUTINE  RECOVER 

' '  THIS  ROUTINE  REASSIGNS  TASKS  TO  WORK  OR  QUEUE  AFTER 
• '  DETERMINATION  OF  PREEMPTION  HAS  BEEN  MADE 
FOR  EACH  TASK  IN  TEMPI 
DO 

REMOVE  THE  FIRST  TASK  FROM  TEMPI 
FILE  THIS  TASK  IN  WORK 
LOOP 

FOR  EACH  TASK  IN  TEMP2 
DO 

REMOVE  THE  FIRST  TASK  FROM  TEMP2 
FILE  THIS  TASK  IN  WORK 
LOOP 

FOR  EACH  TASK  IN  TEMP3 
DO 

REMOVE  THE  FIRST  TASK  FROM  TEMP 3 

LET  SERV. TIME (TASK)  -  NORMAL. F( PI. SERV (TYPE (TASK) ) , 
P2 . SERV (TYPE (TASK) ) #  XO) 

FILE  THIS  TASK  IN  QUEUE 

REMOVE  THIS  EVPOINTER(TASK)  FROM  EV.S(I. E.O.S) 

LOOP 

RETURN 

END 


ROUTINE  CHANGE  GIVEN  DEL. TASK 

• '  THIS  ROUTINE  REDUCES  RESOURCES  BEING  USED  WHEN  A 
' '  PREEMPTED  TASK  IS  REMOVED  FROM  WORK 
DEFINE  DEL. TASK  AS  AN  INTEGER  VARIABLE 
IF  TYPE (DEL. TASK)  =  1 

LET  SERV. TIME (DEL. TASK)  -  EXPONENTIAL. F( PI. SERV(l) ,10) 
ELSE 

LET  SERV. TIME (DEL. TASK)  = 

NORMAL. F( PI. SERV (TYPE (DEL. TASK) ) , 

P2. SERV (TYPE (DEL. TASK) ) ,10) 

ALWAYS 

IF  SERV. TIME (DEL. TASK)  LE  0. 

LET  SERV. TIME (DEL. TASK)  -  0.1  *  REM. TIME (DEL. TASK) 
ALWAYS 

LET  COMP. TIME (DEL. TASK)  -  TIME.V  +  SERV. TIME (DEL. TASK) 

IF  MODAL (DEL. TASK)  =  1 

SUBTRACT  1  FROM  MRM(2,1) 

ELSE 

SUBTRACT  1  FROM  MRM(1,1) 

ALWAYS 

IF  CODE (DEL. TASK)  »  1 

SUBTRACT  1  FROM  MRM(2,2) 

ELSE 

SUBTRACT  1  FROM  MRM(1,2) 

ALWAYS 

IF  STAGE (DEL. TASK)  ■  1 

SUBTRACT  1  FROM  MRM(2.3) 

ELSE 

SUBTRACT  1  FROM  MRM(1,3) 

ALWAYS 

RETURN 

END 

ROUTINE  CORRECT 

• •  THIS  ROUTINE  DECREASES  THE  SERVICE  TIME  OF  TASKS  BEING 
• '  WORKED  WHEN  ANOTHER  TASK  IS  REMOVED  FROM  WORK 
DEFINE  NTASK  AS  AN  INTEGER  VARIABLE 
FOR  EACH  NTASK  IN  WORK 
DO 

LET  REM. TIME (NTASK)  ■  COMP. TIME (NTASK)  -  TIME.V 
LET  SERV. TIME (NTASK)  -  REM. TIME (NTASK)  *  UP. TOTAL 
IF  SERV. TIME (NTASK)  LE  0. 

LET  SERV. TIME (NTASK)  -  0.01  *  REM. TIME (NTASK) 

ALWAYS 

REMOVE  THIS  EVPOINTER (NTASK)  FROM  EV.S(I.E.O.S) 
SCHEDULE  AN  E.O.S (NTASK)  IN  SERV. TIME (NTASK)  SECONDS 
LET  EVPOINTER (NTASK)  -  E.O.S 

LET  COMP. TIME (NTASK)  -  TIME.V  +  SERV. TIME (NTASK) 

LOOP 

FOR  EACH  NTASK  IN  TEMPI 
DO 

LET  REM. TIME (NTASK)  -  COMP. TIME (NTASK)  -  TIME.V 
LET  SERV. TIME (NTASK)  »  REM. TIME (NTASK)  *  UP. TOTAL 


276 


IF  SERV.TIME (NTASK)  LE  0. 

LET  SERV.TIME (NTASK)  *  0.01  *  REM. TIME (NTASK) 
ALWAYS 

REMOVE  THIS  EVPOINTER (NTASK)  FROM  EV.S(I.E.O.S) 
SCHEDULE  AN  E.O.S (NTASK)  IN  SERV.TIME (NTASK)  SECONDS 
LET  EVPOINTER (NTASK)  =  E.O.S 

LET  COMP. TIME (NTASK)  =  TIME.V  +  SERV.TIME (NTASK) 

LOOP 

FOR  EACH  NTASK  IN  TEMP2 
DO 

LET  REM. TIME (NTASK)  -  COMP. TIME (NTASK)  -  TIME.V 
LET  SERV.TIME (NTASK)  ■  REM. TIME (NTASK)  *  UP. TOTAL 
IF  SERV.TIME (NTASK)  LE  0. 

LET  SERV.TIME (NTASK)  -  0.01  *  REM. TIME (NTASK) 
ALWAYS 

REMOVE  THIS  EVPOINTER (NTASK)  FROM  EV.S(I. E.O.S) 
SCHEDULE  AN  E.O.S (NTASK)  IN  SERV.TIME (NTASK)  SECONDS 
LET  EVPOINTER (NTASK)  »  E.O.S 

LET  COMP. TIME (NTASK)  =  TIME.V  +  SERV.TIME (NTASK) 

LOOP 

RETURN 

END 

ROUTINE  DEGRADE  GIVEN  NEW 

• »  THIS  ROUTINE  DETERMINES  THE  DEGRADATION  WHICH  WILL 
' '  RESULT  IF  A  NEW  TASK  IS  ADDED  TO  WORK 
DEFINE  NEW,  OVERLAP,  I,  J,  TOT. RESOURCES  AS  INTEGER 
VARIABLES 
LET  OVERLAP  -  0 
LET  DEG. TOTAL  -  0. 

LET  TOT. RESOURCES  -  0 
FOR  I  -  1  TO  2,  DO 
FOR  J  -  1  TO  3,  DO 

LET  TEST(I,J)  -  MRM(I, J) 

LOOP 

LOOP 

IF  MODAL (NEW)  -  I 
ADD  1  TO  TEST(2 , 1) 

ELSE 

ADD  1  TO  TEST (1,1) 

ALWAYS 

IF  CODE (NEW)  •  1 

ADD  1  TO  TEST (2,2) 

ELSE 

ADD  1  TO  TEST (1,2) 

ALWAYS 

IF  STAGE (NEW)  «  1 
ADD  1  TO  TEST (2,1) 

ELSE 

ADD  1  TO  TEST (1,3) 

ALWAYS 

FOR  I  -  l  TO  2,  DO 
FOR  J  -  1  TO  3,  DO 
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LET  TOT. RESOURCES  *  TOT . RESOURCES  +  TEST(I,  J) 

IF  TEST (I, J)  GT  0 

LET  OVERLAP  *  OVERLAP  +  TEST(I,J)  -  1 
ALWAYS 
LOOP 
LOOP 

IF  OVERLAP  GT  3 

LET  DEG. TOTAL  -  1.1 
ELSE 

IF  OVERLAP  GT  2 

LET  DEG. TOTAL  «  UNIFORM. F( .9,  1.1,  10) 

ELSE 

IF  OVERLAP  GT  1 

LET  DEG. TOTAL  -  UNIFORM. F( .6,  .9,  10) 

ELSE 

IF  OVERLAP  GT  0 

LET  DEG. TOTAL  -  UNIFORM. F(. 3,  .6,  10) 

ELSE 

IF  TOT. RESOURCES  GT  3 

LET  DEG. TOTAL  -  UNIFORM. F( . 1,  .3,  10) 
ALWAYS 
ALWAYS 
ALWAYS 
ALWAYS 
ALWAYS 
RETURN 
END 

ROUTINE  UPGRADE  GIVEN  OLD 

' •  THIS  ROUTINE  DETERMINES  THE  IMPROVEMENT  FACTOR  FOR 
' •  SERVICE  TIME  WHIN  A  TASK  IS  REMOVED  FROM  WORK 
DEFINE  OLD,  OVERLAP,  COUNT,  I,  J,  TOT. RESOURCES, 

OLD. RESOURCES  AS  INTEGER  VARIABLES 
LET  OVERLAP  -  0 
LET  UP. TOTAL  -  0. 

LET  TOT. RESOURCES  -  0 
LET  COUNT  «  0 
LET  FI  -  0. 

LET  F2  -  0. 

LET  OLD. RESOURCES  -  0 
FOR  I  -  1  TO  2,  DO 
FOR  J  -  1  TO  3,  DO 

LET  OLD. RESOURCES  -  OLD . RESOURCES  +  KRH(I,J) 

IF  MRM(I,J)  GT  0 

LET  COUNT  -  COUNT  +  MRM(I,U)  -  1 
ALWAYS 
LOOP 
LOOP 

IF  COUNT  GT  3 
LET  FI  -  1.1 
ELSE 

IF  COUNT  GT  2 

LET  FI  -  UNIFORM. F(. 9,  1.1,  10) 
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ELSE 

IF  COUNT  GT  1 

LET  FI  =  UNIFORM. F( .6,  .9,  10) 

ELSE 

IF  COUNT  GT  0 

LET  FI  -  UNIFORM. F(. 3,  . 6 ,  10) 

ELSE 

IF  OLD. RESOURCES  GT  3 

LET  FI  »  UNIFORM. F(.l,  .3,  10) 

ALWAYS 

ALWAYS 

ALWAYS 

ALWAYS 

ALWAYS 

FOR  I  »  1  TO  2,  DO 
FOR  J  -  1  TO  3,  DO 

LET  TEST (  X ,  J )  -  MRM(I,J) 

LOOP 

LOOP 

IF  MODAL (OLD)  «  1 

SUBTRACT  1  FROM  TEST (2,1) 

ELSE 

SUBTRACT  1  FROM  TEST (1,1) 

ALWAYS 

IF  CODE (OLD)  •  1 

SUBTRACT  1  FROM  TEST (2, 2) 

ELSE 

SUBTRACT  1  FROM  TEST (1,2) 

ALWAYS 

IF  STAGE (OLD)  -  1 

SUBTRACT  1  FROM  TEST (2, 3) 

ELSE 

SUBTRACT  1  FROM  TEST (1,3) 

ALWAYS 

FOR  I  -  l  TO  2,  DO 
FOR  J  -  1  TO  3,  DO 

LET  TOT. RESOURCES  -  TOT . RESOURCES  +  TEST(I,J) 
IF  TEST(I,J)  GT  0 

LET  OVERLAP  -  OVERLAP  +  TEST(I,J)  -  1 
ALWAYS 
LOOP 
LOOP 

IF  OVERLAP  GT  3 
LET  F2  -  1.1 
ELSE 

IF  OVERLAP  GT  2 

LET  F2  -  UNIFORM. F(. 9,  1.1,  10) 

ELSE 

IF  OVERLAP  GT  1 

LET  F2  -  UNIFORM. F(. 6,  .9,  10) 

ELSE 

IF  OVERLAP  GT  0 
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LET  F2  =  UNIFORM. F( . 3,  .6,  10) 

ELSE 

IF  TOT. RESOURCES  GT  3 

LET  F2  *  UNIFORM. F(.l,  .3,  10) 

ALWAYS 

ALWAYS 

ALWAYS 

ALWAYS 

ALWAYS 

LET  UP. TOTAL  *  Fl/(1  +  FI) 

RETURN 

END 

EVENT  E.O.S(TASK) 

• '  THIS  EVENT  DESTROYS  A  TASK  WHICH  HAS  COMPLETED  SERVICE 

DEFINE  TASK,  KEY  AS  INTEGER  VARIABLES 

CALL  UPGRADE  GIVEN  TASK 

CALL  CHANGE  GIVEN  TASK 

IF  TYPE (TASK)  -  7 

SCHEDULE  AN  H. ARRIVAL  NOW 
ALWAYS 

REMOVE  THE  FIRST  TASK  FROM  WORK 
DESTROY  THIS  TASK  CALLED  TASK 
CALL  CORRECT 
IF  QUEUE  IS  NOT  EMPTY 
FOR  EACH  TASK  IN  QUEUE 
DO 

IF  TYPE (TASK)  -  12 

IF  TIME.V  -  A. TIKE (TASK)  GT  50.0 
ADD  1  TO  PRIOR (TASK) 

ALWAYS 

ELSE 

IF  TIME.V  -  A. TIME (TASK)  GT  10.0 
ADD  1  TO  PRIOR (TASK) 

ALWAYS 

ALWAYS 

CALL  DEGRADE  GIVEN  TASK 
IF  DEG. TOTAL  LT  1.0 

LET  COMP. TIKE (TASK)  -  TIME.V  +  SERV. TIME (TASK) 
REMOVE  THE  FIRST  TASK  FROM  QUEUE 
LET  KEY  «  0 

CALL  ADJUST  GIVEN  TASK,  KEY 
ELSE 

IF  PRIOR(TASK)  GT  1 
LET  KEY  *  1 

CALL  PREEMPT  GIVEN  TASK,  KEY 
ALWAYS 
ALWAYS 
LOOP 
ALWAYS 
CALL  STATUS 
RETURN 
END 
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EVENT  END. SIM 

' •  THIS  EVENT  PROVIDES  TERMINAL  STATISTICS  AND  ENDS 
"  SIMULATION 

DEFINE  I,  J,  THING  AS  INTEGER  VARIABLES 
LET  W.IDLE  -  1  -  AV . BUSY 
LET  SIM. 3  -  SIM. TOT/ ARR . TOT 
LET  Pl.ARR(l)  -  2.083*P1.ARR(1) 

LET  PI. ARR (2)  -  1.333*P1.ARR(2) 

LET  PI. ARR (3)  *  1.667*P1.ARR(3) 

LET  P1.ARR(4)  *  1. 667*P1 .ARR(4) 

LET  P1.ARR(5)  -  1.667*P1.ARR(5) 

LET  PI. ARR(6)  *  1.429*P1.ARR(6) 

LET  P1.ARR(9)  -  1.538*P1.ARR(9) 

LET  TIME.V  *  0. 

LET  BUSY  «  0 

RESET  THE  TOTALS  OF  NUM. ARRIVE,  BUSY,  SIMUL 
UNTIL  EV.S (I .A. ARRIVAL)  IS  EMPTY,  DO 

REMOVE  FIRST  THING  FROM  EV.S (I. A. ARRIVAL) 

DESTROY  A. ARRIVAL  CALLED  THING 
LOOP 

UNTIL  EV. S( I. B. ARRIVAL)  IS  EMPTY,  DO 

REMOVE  FIRST  THING  FROM  EV. S( I. B. ARRIVAL) 

DESTROY  B. ARRIVAL  CALLED  THING 
LOOP 

UNTIL  EV. Sf I. C. ARRIVAL)  IS  EMPTY,  DO 

REMOVE  FIRST  THING  FROM  EV.S ( I. C. ARRIVAL) 

DESTROY  C. ARRIVAL  CALLED  THING 
LOOP 

UNTIL  EV.S (I. D. ARRIVAL)  IS  EMPTY,  DO 

REMOVE  FIRST  THING  FROM  EV. S( I. D. ARRIVAL) 

DESTROY  D. ARRIVAL  CALLED  THING 
LOOP 

UNTIL  EV. S( I. E. ARRIVAL)  IS  EMPTY,  DO 

REMOVE  FIRST  THING  FROM  EV. S( I. E. ARRIVAL) 

DESTROY  E. ARRIVAL  CALLED  THING 
LOOP 

UNTIL  EV. S( I. F. ARRIVAL)  IS  EMPTY,  DO 

REMOVE  FIRST  THING  FROM  EV. S( I. F. ARRIVAL) 

DESTROY  F. ARRIVAL  CALLED  THING 
LOOP 

UNTIL  EV.S ( I. G. ARRIVAL)  IS  EMPTY,  DO 

REMOVE  FIRST  THING  FROM  EV. S( I. G. ARRIVAL) 

DESTROY  G. ARRIVAL  CALLED  THING 
LOOP 

UNTIL  EV.S ( I. H. ARRIVAL)  IS  EMPTY,  DO 

REMOVE  FIRST  THING  FROM  EV.S (I .H. ARRIVAL) 

DESTROY  H. ARRIVAL  CALLED  THING 
LOOP 

UNTIL  EV.S (I. I. ARRIVAL)  IS  EMPTY,  DO 

REMOVE  FIRST  THING  FROM  EV.S (I. I. ARRIVAL) 

DESTROY  I. ARRIVAL  CALLED  THING 
LOOP 


281 


UNTIL  EV.S ( I. J. ARRIVAL)  IS  EMPTY,  DO 

REMOVE  FIRST  THING  FROM  EV.S ( I. J. ARRIVAL) 

DESTROY  J. ARRIVAL  CALLED  THING 
LOOP 

UNTIL  EV.S (I. K. ARRIVAL)  IS  EMPTY,  DO 
00  0 

REMOVE  FIRST  THING  FROM  EV.S (I. K. ARRIVAL) 

DESTROY  K. ARRIVAL  CALLED  THING 
LOOP 

UNTIL  EV.S (I. L. ARRIVAL)  IS  EMPTY,  DO 

REMOVE  FIRST  THING  FROM  EV.S (I. L. ARRIVAL) 

DESTROY  L. ARRIVAL  CALLED  THING 
LOOP 

UNTIL  EV.S (I. E.O.S)  IS  EMPTY,  DO 

REMOVE  FIRST  THING  FROM  EV.S(I. E.O.S) 

DESTROY  E.O.S  CALLED  THING 
LOOP 

UNTIL  WORK  IS  EMPTY,  DO 

REMOVE  FIRST  TASK  FROM  WORK 
DESTROY  THIS  TASK  CALLED  TASK 
LOOP 

UNTIL  QUEUE  IS  EMPTY,  DO 

REMOVE  FIRST  TASK  FROM  QUEUE 
DESTROY  THIS  TASK  CALLED  TASK 
LOOP 

UNTIL  TEMPI  IS  EMPTY,  DO 

REMOVE  FIRST  TASK  FROM  TEMPI 
DESTROY  THIS  TASK  CALLED  TASK 
LOOP 

UNTIL  TEMP2  IS  EMPTY,  DO 

REMOVE  FIRST  TASK  FROM  TEMP2 
DESTROY  THIS  TASK  CALLED  TASK 
LOOP 

UNTIL  TEMPI  IS  EMPTY,  DO 

REMOVE  FIRST  TASK  FROM  TEMPI 
DESTROY  THIS  TASK  CALLED  TASK 
LOOP 

FOR  I  -  1  TO  2,  DO 
FOR  J  -  1  TO  3,  DO 
LET  NRM(I,  J)  «  0 
LET  TEST(I,J)  -  0 
LOOP 
LOOP 
RETURN 
END 

EVENT  DATA. RESET 

• '  THIS  EVENT  RESETS  DATA  COUNTERS  AT  A  SPECIFIC  TIME 

RESET  THE  TOTALS  OF  NUN. ARRIVE,  SIKUL,  BUSY 

RETURN 

END 

ROUTINE  SNAP.R 

DEFINE  I,  J  AS  INTEGER  VARIABLES 


FOR  EACH  TASK  IN  QUEUE,  LIST  ATTRIBUTES  OF  TASK 

FOR  EACH  TASK  IN  WORK,  LIST  ATTRIBUTES  OF  TASK 

FOR  EACH  TASK  IN  TEMPI,  LIST  ATTRIBUTES  OF  TASK 

FOR  EACH  TASK  IN  TEMP2,  LIST  ATTRIBUTES  OF  TASK 

FOR  EACH  TASK  IN  TEMP3,  LIST  ATTRIBUTES  OF  TASK 

LIST  ATTRIBUTES  OF  EACH  A. ARRIVAL  IN  EV.S(I.A.ARRIVAL) 
LIST  ATTRIBUTES  OF  EACH  B. ARRIVAL  IN  EV. S( I. B. ARRIVAL) 
LIST  ATTRIBUTES  OF  EACH  C. ARRIVAL  IN  EV. S( I. C. ARRIVAL) 
LIST  ATTRIBUTES  OF  EACH  D. ARRIVAL  IN  EV. S( I, D. ARRIVAL) 
LIST  ATTRIBUTES  OF  EACH  E. ARRIVAL  IN  EV . S ( I . E . ARRIVAL) 
LIST  ATTRIBUTES  OF  EACH  F. ARRIVAL  IN  EV. S( I. F. ARRIVAL) 
LIST  ATTRIBUTES  OF  EACH  G. ARRIVAL  IN  EV.S (I. G. ARRIVAL) 
LIST  ATTRIBUTES  OF  EACH  H.  ARRIVAL  IN  EV.S  (I. H. ARRIVAL) 
LIST  ATTRIBUTES  OF  EACH  I. ARRIVAL  IN  EV.S (I. I. ARRIVAL) 
LIST  ATTRIBUTES  OF  EACH  J. ARRIVAL  IN  EV. S ( I. J. ARRIVAL) 
LIST  ATTRIBUTES  OF  EACH  K. ARRIVAL  IN  EV. S (I. K. ARRIVAL) 
LIST  ATTRIBUTES  OF  EACH  L. ARRIVAL  IN  EV.S ( I. L. ARRIVAL) 
LIST  ATTRIBUTES  OF  EACH  E.O.S  IN  EV.S(I.E.O.S) 

LIST  TIME.V,  N. WORK,  N. QUEUE,  BUSY,  N. TEMPI,  N.TEMP2, 
N.TEMP3,  DEG. TOTAL,  UP. TOTAL 
FOR  I  «  1  TO  2,  DO 
FOR  J  -  X  TO  3,  DO 
LIST  MRM(I,J) 

LOOP 

LOOP 

SKIP  2  LINES 

RETURN 

END 
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WORKLOAD  SURVEY 

Many  jobs  can  be  described  as  multi-task,  situations. 
These  jobs  are  characterized  by  a  variety  of  repetitive 
tasks  which  demand  an  individual's  attention.  Common 
examples  of  these  multi-task  situations  include  driving  a 
car  in  heavy  traffic,  operating  a  nuclear  power  plant, 
flying  an  aircraft,  or  being  a  command  post  controller  or 
an  air  traffic  controller. 

In  a  multi-task  environment,  tasks  may  arrive  at  random 
times  so  that  an  operator's  workload  varies.  The  time 
required  to  complete  each  task  may  also  be  random.  These 
tasks  may  arrive  so  one  task  is  completed  before  another 
arrives,  or  they  may  overlap  so  that  more  than  one  task  is 
present  at  a  time.  In  some  cases  more  than  one  task  may  be 
accomplished  at  a  time,  while  in  other  cases  a  single  task 
may  require  an  operator's  complete  attention.  Some  tasks 
may  have  a  higher  priority  than  others  and  may  interrupt 
lower  priority  tasks.  If  a  task  cannot  interrupt  another, 
and  cannot  be  accomplished  simultaneously,  it  must  wait 
until  an  operator  is  free  to  work  on  that  task. 

Assume  that  you  are  the  operator  in  a  multi-task 
environment  as  just  described.  Your  WORKLOAD  could  be 
described  as  LOW,  MEDIUM,  or  HIGH  and  characterized  as 
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follows: 

LOW  WORKLOAD: 

-  No,  or  vary  few,  interruptions  in  the  planning, 
execution,  or  monitoring  of  tasks*  Spare  time 
exists  between  many  tasks. 

MEDIUM  WORKLOAD: 

-  Task  planning,  execution  and  monitoring  are 
often  interrupted.  Little  spare  time.  Tasks 
occasionally  occur  simultaneously. 

HIGH  WORKLOAD: 

-  Task  planning,  execution  and  monitoring  are 
interrupted  most  of  the  time.  Very  little  spare 
time.  Tasks  frequently  occur  simultaneously. 
Considerable  difficulty  in  accomplishing  all 
tasks, 

I  would  like  you  to  translate  these  categorical  ratings 
to  numerical  ratings.  Assume  that  three  measures  of  merit 
have  been  developed  as  surrogates  for  workload.  That  is, 
assume  workload  cannot  be  measured  directly  so  three  other 
measures  have  been  identified  as  possible  surrogates,  or 
replacements.  These  measures  are  Spare  Time,  Task 
Interruption  Rate,  and  simultaneous  Task  Rate. 

First,  consider  Spare  Time  which  is  the  percentage  of 
time  when  you  are  not  working  on  any  tasks.  For  each  of 


286 


the  three  categories  described,  what  range  of  Spare  Time 
would  you  associate  with  it? 

HIGH  WORKLOAD:  0  %3  [  SPARE  TIME  ]  3  _  % 

MEDIUM  WORKLOAD:  _  %3  [  SPARE  TIME  ]  3  _  % 

LOW  WORKLOAD:  _  %3  [  SPARE  TIME  ]  3  100  % 

Next,  consider  the  Task  Interruption  Rate.  This  is  the 
percentage  of  tasks  interrupted  at  least  once  before 
completion.  It  is  fair  to  assume  that  the  more  often  tasks 
are  interrupted,  the  higher  your  workload  will  be. 

HIGH  WORKLOAD:  _ _  *3  [  TASK  INTERRUPT  RATE  ]  3  100  % 

MEDIUM  WORKLOAD:  _  %3  [  TASK  INTERRUPT  RATE  ]  3  _  % 

LOW  WORKLOAD:  0  %3  [  TASK  INTERRUPT  RATE  ]  S  _  % 

Finally,  consider  the  simultaneous  Task  Rate.  This  is 
the  percentage  of  tasks  that  will  be  accomplished 
simultaneously  with  another  task.  It  is  fair  to  assume 
that  a  high  Simultaneous  Task  Rate  corresponds  to  high 
workload. 


HIGH  WORKLOAD: 

_  ** 

C 

SIMULTANEOUS 

TASK 

RATE 

] 

s 

100  % 

MEDIUM  WORKLOAD: 

_ 

C 

SIMULTANEOUS 

TASK 

RATE 

1 

3 

% 

LOW  WORKLOAD :■ 

0  %3 

C 

SIMULTANEOUS 

TASK 

RATE 

] 

3 

% 
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What  is  your  age?  _ 

Circle  one:  HALE  FEMALE 

Are  you  a  pilot?  YES  NO 

Are  you  familiar  with  any  of  the  multi-task  situations 
described 

in  the  first  paragraph?  If  so,  which  ones? 


Thank  you  for  your  time  and  cooperation. 
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Table  31  Simulation  Output,  SWAT  Score  Transformations 
(All  Responses) 


SIMULATION 

OUTPUT 

PARAMETER 

PARAMETER  RANGES 

SWAT  CONVERSION 

PILOT  IDLE 

0%  <  PIR  <  12,3% 

HIGH  (3) 

RATE  (PIR) 

12.3%  <  PIR  <  44.1% 

MEDIUM  (2) 

44.1%  <  PIR  <  100% 

LOW  (1) 

TASK  INTERRUPT 

69.3%  <  TIR  <  100% 

HIGH  (3) 

RATE  (TIR) 

28.9%  <  TIR  <  69,3% 

MEDIUM  (2) 

0%  <  TIR  <  28.9% 

LOW  (1) 

SIMULTANEOUS 

65.5%  <  STR  <  100% 

HIGH  (3) 

TASK  RATE 

26.2%  <  STR  <  65.5% 

MEDIUM  (2) 

(STR) 

0%  <  STR  <  26.2% 

LOW  (1) 

Table  32  Simulation  Output,  SWAT  Score  Transformations 
(Pilots  Only) 


SIMULATION 

OUTPUT 

PARAMETER 

PARAMETER  RANGES 

SWAT  CONVERSION 

PILOT  IDLE 

0%  <  PIR  <  11.1% 

HIGH  (3) 

RATE  (PIR) 

11.1%  <  PIR  <  42.1% 

MEDIUM  (2) 

42.1%  <  PIR  <  100% 

LOW  (1) 

TASK  INTERRUPT 

68.6%  <  TIR  <  100% 

HIGH  (3) 

RATE  (TIR) 

27.9%  <  TIR  <  68,6% 

MEDIUM  (2) 

0%  <  TIR  <  27.9% 

LOW  (1) 

SIMULTANEOUS 

65.3%  <  STR  <  100% 

HIGH  (3) 

TASK  RATE 

25.1%  <  STR  <  65.3% 

MEDIUM  (2) 

(STR) 

0%  <  STR  <  25.1% 

LOW  (1) 

Table  33  Simulation  Output,  SWAT  Score  Transformations 
(Experts  Only) 


SIMULATION 

OUTPUT 

PARAMETER 

PARAMETER  RANGES 

SWAT  CONVERSION 

PILOT  IDLE 

0%  <  PIR  <  17.5% 

HIGH  (3) 

RATE  (PIR) 

17.5%  <  PIR  <  65.5% 

MEDIUM  (2) 

65.5%  <  PIR  <  100% 

LOW  (1) 

TASK  INTERRUPT 

73.8%  <  TIR  <  100% 

HIGH  (3) 

RATE  (TIR) 

37.5%  <  TIR  <  73.8% 

MEDIUM  (2) 

0%  <  TIR  <  37.5% 

LOW  (1) 

SIMULTANEOUS 

67.5%  <  STR  <  100% 

HIGH  (3) 

TASK  RATE 

31.3%  <  STR  <  67.5% 

MEDIUM  (2) 

(STR) 

0%  <  STR  <  31.3% 

LOW  (1) 

APPENDIX  H 


WORKLOAD  RATINGS  AND  PREDICTIONS 
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This  appendix  contains  a  complete  listing  of  all  workload 


ratings  and  predictions.  These  numerical  values  are  the 
SWAT  ratings,  scaled  between  0  and  100,  for  the  three 
flight  segments:  penetration  (PEN) ,  target  acquisition 
(ACQ) ,  and  weapon  delivery  (WPN) .  Each  set  of  workload 
ratings  are  categorized  by  predictor  type  (PRED) ,  cockpit 
configuration  (CONFIG) ,  and  the  original  method  used  to 
develop  the  predictors  (METHOD) .  Definitions  of  terms  used 
in  this  Appendix  are  as  follows: 

PREDICTORS: 

o  SWAT  -  Data  derived  from  workload  measurements  or 
estimates  and  not  modified  by  simulation  output, 
o  SSI  -  Workload  estimates  developed  by  using  pilot  idle 
time  as  the  surrogate  Time  Load  predictor,  as  determined 
from  the  serial  processing  model. 

o  SSTA  -  Workload  estimates  developed  by  using  task 
interrupt  rate,  serial  processing  model,  where  task 
preempts  are  counted  as  interrupts  and  subsequent  arrivals 
of  preempted  tasks  are  counted  as  new  arrivals, 
o  SSTB  -  Workload  estimates  developed  by  using  task 
interrupt  rate,  serial  processing  model,  where  task 
preempts  are  counted  as  interrupts  and  subsequent  arrivals 
of  preempted  tasks  are  not  counted  as  new  arrivals, 
o  SSTC  -  Workload  estimates  developed  by  using  task 
interrupt  rate,  serial  processing  model,  where  the  arrival 
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of  a  new  task  while  at  least  one  other  task  is  being  worked 
is  counted  as  a  task  interruption.  Subsequent  arrivals  of 
preempted  tasks  are  counted  as  new  arrivals, 
o  SSTD  -  Workload  estimates  developed  by  using  task 
interrupt  rate,  serial  processing  model,  where  the  arrival 
of  a  new  task  while  at  least  one  other  task  is  being  worked 
is  counted  as  a  task  interruption.  Subsequent  arrivals  of 
preempted  tasks  are  not  counted  as  new  arrivals, 
o  KRMX  -  Workload  estimates  developed  by  using  pilot  idle 
time  as  the  surrogate  Time  Load  predictor,  as  determined 
from  the  parallel  processing  model, 
o  KRMS  -  Workload  estimates  developed  by  using  task 
interrupt  rate  as  the  surrogate  Time  Load  predictor,  as 
determined  by  the  parallel  processing  model. 

CONFIGURATIONS ; 
o  ABL  -  Advanced  Baseline 
o  TFTA  -  Terrain  Following/Terrain  Avoidance 
o  TTA  -  Terrain/Threat  Avoidance 

METHODS i 

o  SWAT  -  RAM/ ACE  SWAT  workload  measurements  (1982) 
o  PRO  -  RAM/ACE  SWAT  workload  estimates  (1982) 
o  SCK  -  PRO-SWAT  workload  estimates  (1986) 


t  »  i  Ml  MT — te — — -V- 
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Table  34  Workload  Measurements  and  Estimates 


PRED 

PEN 

ACQ 

WPN 

CONFIG  METHOD 

SSI 

65.9 

40.8 

45.2 

ABL 

SCH 

SSI 

59.9 

30.0 

100.0 

ABL 

SCH 

SSI 

45.7 

40.8 

95.1 

ABL 

SCH 

SSI 

52.2 

52.2 

69.4 

ABL 

SCH 

SSI 

59.9 

76.3 

83.6 

ABL 

SCH 

SSI 

52.2 

72.0 

69.4 

ABL 

SCH 

SSI 

59.0 

82.8 

69.4 

ABL 

SCH 

SSI 

10.4 

52.2 

40.6 

ABL 

SCH 

SSI 

26.2 

40.8 

16.0 

TFTA 

SCH 

SSI 

70.4 

30.0 

45.7 

TFTA 

SCH 

SSI 

65.9 

16.0 

65.9 

TFTA 

SCH 

SSI 

52.2 

39.3 

52.2 

TFTA 

SCH 

SSI 

76.3 

76.3 

59.9 

TFTA 

SCH 

SSI 

39.2 

52.2 

52.2 

TFTA 

SCH 

SSI 

72.0 

63.1 

10.4 

TFTA 

SCH 

SSI 

52.2 

10.4 

10.4 

TFTA 

SCH 

SSI 

16.0 

40.8 

1.4 

TTA 

SCH 

SSI 

59.9 

30.0 

76.3 

TTA 

SCH 

SSI 

40.8 

16.0 

40.8 

TTA 

SCH 

SSI 

23.5 

39.2 

52.2 

TTA 

SCH 

SSI 

59.9 

59.9 

59.9 

TTA 

SCH 

SSI 

39.2 

52.2 

52.2 

TTA 

SCH 

SSI 

59.0 

63.1 

10.4 

TTA 

SCH 

SSI 

52.2 

52.2 

52.2 

TTA 

SCH 

SSTD 

8.0 

8.0 

86.2 

ABL 

PRO 

SSTD 

71.2 

71.2 

100.0 

ABL 

PRO 

SSTD 

21.2 

45.6 

86.2 

ABL 

PRO 

SSTD 

71.2 

84.9 

100.0 

ABL 

PRO 

SSTD 

71.2 

45.6 

86.2 

ABL 

PRO 

SSTD 

0.0 

8.0 

24.3 

TFTA 

PRO 

SSTD 

35.6 

21.2 

60.0 

TFTA 

PRO 

SSTD 

0.0 

45.6 

60.0 

TFTA 

PRO 

SSTD 

61.9 

45.6 

60.0 

TFTA 

PRO 

SSTD 

13.7 

58.0 

86.2 

TFTA 

PRO 

SSTD 

21.2 

45.6 

46.8 

TTA 

PRO 

SSTD 

21.2 

45.6 

100.0 

TTA 

PRO 

SSTD 

45.6 

21.2 

37.5 

TTA 

PRO 

SSTD 

21.2 

21.2 

100.0 

TTA 

PRO 

SSTD 

8.0 

31.8 

86.2 

TTA 

PRO 

SSTB 

0.0 

0.0 

61.9 

ABL 

PRO 

SSTB 

61.9 

61.9 

74.3 

ABL 

PRO 

SSTB 

13.7 

35.6 

61.9 

ABL 

PRO 

SSTB 

61.9 

74.3 

74.3 

ABL 

PRO 

SSTB 

61.9 

35.6 

61.9 

ABL 

PRO 

SSTB 

0.0 

0.0 

0.0 

TFTA 

PRO 

SSTB 

35.6 

13.7 

35.6 

TFTA 

PRO 
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Table  34  Workload  Measurements  and  Estimates  (Continued) 


PEED 

PEN 

ACQ 

WPN 

CONFIG  METHOD 

SSTB 

0.0 

35.6 

35.6 

TFTA 

PRO 

SSTB 

61.9 

35.6 

35.6 

TFT  A 

PRO 

SSTB 

13.7 

48.8 

61.9 

TFTA 

PRO 

SSTB 

13.7 

35.6 

21.7 

TTA 

PRO 

SSTB 

13.7 

35.6 

74.3 

TTA 

PRO 

SSTB 

35.6 

13.7 

13.7 

TTA 

PRO 

SSTB 

13.7 

13.7 

74.3 

TTA 

PRO 

SSTB 

0.0 

21.7 

61.9 

TTA 

PRO 

SSTB 

64.5 

39.3 

14.6 

ABL 

SCH 

SSTB 

29.9 

0.0 

46.3 

ABL 

SCH 

SSTB 

44.2 

39.3 

64.5 

ABL 

SCH 

SSTB 

41.8 

41.8 

41.8 

ABL 

SCH 

SSTB 

29.9 

46.3 

29.9 

ABL 

SCH 

SSTB 

41.8 

61.5 

41.8 

ABL 

SCH 

SSTB 

48.5 

72.4 

41.8 

ABL 

SCH 

SSTB 

0.0 

41.8 

13.0 

ABL 

SCH 

SSTB 

24.7 

39.3 

14.6 

TFTA 

SCH 

SSTB 

40.4 

0.0 

15.7 

TFTA 

SCH 

SSTB 

64.5 

14.6 

64.5 

TFTA 

SCH 

SSTB 

41.8 

28.8 

41.8 

TFTA 

SCH 

SSTB 

46.3 

46.3 

29.9 

TFTA 

SCH 

SSTB 

28.8 

41.8 

41.8 

TFTA 

SCH 

SSTB 

61.5 

52.6 

0.0 

TFTA 

SCH 

SSTB 

41.8 

0.0 

0.0 

TFTA 

SCH 

SSTB 

14.6 

39.3 

0.0 

TTA 

SCH 

SSTB 

29.9 

0.0 

46.3 

TTA 

SCH 

SSTB 

39.3 

14.6 

39.3 

TTA 

SCH 

SSTB 

13.0 

28.8 

41.8 

TTA 

SCH 

SSTB 

29.9 

29.9 

29.9 

TTA 

SCH 

SSTB 

28.8 

41.8 

41.8 

TTA 

SCH 

SSTB 

48.5 

52.6 

0.0 

TTA 

SCH 

SSTB 

41.8 

41.8 

41.8 

TTA 

SCH 

SSTC 

8.0 

24.3 

86.2 

ABL 

PRO 

SSTC 

71.2 

86.2 

100.0 

ABL 

PRO 

SSTC 

21.2 

60.0 

86.2 

ABL 

PRO 

SSTC 

71.2 

100.0 

100.0 

ABL 

PRO 

SSTC 

71.2 

60.0 

86.2 

ABL 

PRO 

SSTC 

8.0 

8.0 

24.3 

TFTA 

PRO 

SSTC 

45.6 

21.2 

60.0 

TFTA 

PRO 

SSTC 

8.0 

45.6 

60.0 

TFTA 

PRO 

SSTC 

71.2 

45.6 

60.0 

TFTA 

PRO 

SSTC 

21.2 

58.0 

86.2 

TFTA 

PRO 

SSTC 

21.2 

45.6 

46.8 

TTA 

PRO 

SSTC 

21.2 

45.6 

100.0 

TTA 

PRO 

SSTC 

45.6 

21.2 

37.5 

TTA 

PRO 

SSTC 

21.2 

21.2 

100.0 

TTA 

PRO 
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Table  34  Workload  Measurements  and  Estimates  (Continued) 


PRED 

PEN 

ACQ 

WPN 

CONFIG 

METHOD 

SSTC 

8.0 

31.8 

86.2 

TTA 

PRO 

MRMI 

64.5 

40.8 

45.2 

ABL 

SCH 

MRMI 

29.9 

30.0 

100.0 

ABL 

SCH 

MRMI 

44.2 

40.8 

65.9 

ABL 

SCH 

MRMI 

41.8 

52.2 

69.4 

ABL 

SCH 

MRMI 

29.9 

76.3 

83.6 

ABL 

SCH 

MRMI 

41.8 

72.0 

69.4 

ABL 

SCH 

MRMI 

48.5 

32.2 

69.4 

ABL 

SCH 

MRMI 

0.0 

52.2 

40.6 

ABL 

SCH 

MRMI 

24.7 

40.8 

16.0 

TFTA 

SCH 

MRMI 

40.4 

30.0 

45.7 

TFT  A 

SCH 

MRMI 

64.5 

16.0 

65.9 

TFTA 

SCH 

MRMI 

41.8 

39.2 

52.2 

TFTA 

SCH 

MRMI 

46.3 

76.3 

59.9 

TFTA 

SCH 

MRMI 

28.8 

52.2 

52.2 

TFTA 

SCH 

MRMI 

61.5 

63.1 

10.4 

TFTA 

SCH 

MRMI 

41.8 

10.4 

10.4 

TFTA 

SCH 

MRMI 

14.6 

40.8 

1.4 

TTA 

SCH 

MRMI 

29.9 

30.0 

76.3 

TTA 

SCH 

MRMI 

39.3 

16.0 

40.8 

TTA 

SCH 

MRMI 

13.0 

39.2 

52.2 

TTA 

SCH 

MRMI 

29.9 

59.9 

59.9 

TTA 

SCH 

MRMI 

28.8 

52.2 

52.2 

TTA 

SCH 

MRMI 

48.5 

63.1 

10.4 

TTA 

SCH 

MRMI 

41.8 

52.2 

52.2 

TTA 

SCH 

MRMS 

65.9 

70.0 

45.2 

ABL 

SCH 

MRMS 

59.9 

53.7 

100.0 

ABL 

SCH 

MRMS 

45.7 

70.0 

65.9 

ABL 

SCH 

MRMS 

52.2 

69.4 

69.4 

ABL 

SCH 

MRMS 

59.9 

100.0 

83.6 

ABL 

SCH 

MRMS 

52.2 

89.2 

69.4 

ABL 

SCH 

MRMS 

59.0 

100.0 

69.4 

ABL 

SCH 

MRMS 

10.4 

69.4 

40.6 

ABL 

SCH 

MRMS 

26.2 

40.8 

45.2 

TFTA 

SCH 

MRMS 

70.4 

30.0 

69.4 

TFTA 

SCH 

MRMS 

65.9 

16.0 

95.1 

TFTA 

SCH 

MRMS 

52.2 

39.2 

69.4 

TFTA 

SCH 

MRMS 

76.3 

76.3 

83.6 

TFTA 

SCH 

MRMS 

39.2 

52.2 

69.4 

TFTA 

SCH 

MRMS 

72.0 

63.1 

27.6 

TFTA 

SCH 

MRMS 

52.2 

10.4 

27.6 

TFTA 

SCH 

MRMS 

16.0 

40.8 

30.6 

TTA 

SCH 

MRMS 

59.9 

30.0 

100.0 

TTA 

SCH 

MRMS 

40.8 

16.0 

70.0 

TTA 

SCH 

MRMS 

23.5 

39.2 

69.4 

TTA 

SCH 

MRMS 

29.9 

59.9 

83.6 

TTA 

SCH 
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Table  34  Workload  Measurements  and  Estimates  (Continued) 


PRED 

PEN 

ACQ 

WPN 

CONFIG 

METHOD 

MRMS 

39.2 

52.2 

69.4 

TTA 

SCH 

MRMS 

59.0 

63.1 

27.6 

TTA 

SCH 

MRMS 

52.2 

52.2 

69.4 

TTA 

SCH 

SSTD 

65.9 

40.8 

45.2 

ABL 

SCH 

SSTD 

59.9 

30.0 

100.0 

AST, 

SCH 

SSTD 

45.7 

40.8 

95.1 

ABL 

SCH 

SSTD 

52.2 

52.2 

69.4 

ABL 

SCH 

SSTD 

59.9 

76.3 

83.6 

ABL 

SCH 

SSTD 

52.2 

72.0 

69.4 

ABL 

SCH 

SSTD 

59.0 

82.8 

69.4 

ABL 

SCH 

SSTD 

10.4 

52.2 

40.6 

ABL 

SCH 

SSTD 

24.7 

40.8 

45.2 

TFTA 

SCH 

SSTD 

40.4 

30.0 

69.4 

TFT  A 

SCH 

SSTD 

64.5 

16.0 

95.1 

TFTA 

SCH 

SSTD 

41.8 

39.3 

69.4 

TFTA 

SCH 

SSTD 

46.3 

76.3 

83.6 

TFTA 

SCH 

SSTD 

28.8 

52.2 

69.4 

TFTA 

SCH 

SSTD 

61.5 

63.1 

27.6 

TFTA 

SCH 

SSTD 

41.8 

10.4 

27.6 

TFTA 

SCH 

SSTD 

16.0 

40.8 

30.6 

TTA 

SCH 

SSTD 

59.9 

30.0 

100.0 

TTA 

SCH 

SSTD 

40.8 

16.0 

70,0 

TTA 

SCH 

SSTD 

23.5 

39.2 

69.4 

TTA 

SCH 

SSTD 

59.9 

59.9 

83.6 

TTA 

SCH 

SSTD 

39.2 

52.2 

69.4 

TTA 

SCH 

SSTD 

59.0 

63.1 

27.6 

TTA 

SCH 

SSTD 

52.2 

52.2 

69.4 

TTA 

SCH 

SSTC 

65.9 

70.0 

45.2 

ABL 

SCH 

SSTC 

59.9 

53.7 

100.0 

ABL 

SCH 

SSTC 

45.7 

70.0 

95.1 

ABL 

SCH 

SSTC 

52.2 

69.4 

59.4 

ABL 

SCH 

SSTC 

59.9 

100.0 

83.6 

ABL 

SCH 

SSTC 

52.2 

89.2 

69.4 

ABL 

SCH 

SSTC 

59. 0 

100.0 

69.4 

ABL 

SCH 

SSTC 

10.4 

69.4 

40.6 

ABL 

SCH 

SSTC 

26.2 

40.8 

45.2 

TFTA 

SCH 

SSTC 

70.4 

30.0 

69.4 

TFTA 

SCH 

SSTC 

65.9 

16.0 

95.1 

TFTA 

SCH 

SSTC 

52.2 

39.3 

69.4 

TFTA 

SCH 

SSTC 

76.3 

76.3 

83.6 

TFTA 

SCH 

SSTC 

39.2 

52.2 

69.4 

TFTA 

SCH 

SSTC 

72.0 

63.1 

27.6 

TFTA 

SCH 

SSTC 

52.2 

10.4 

27.6 

TFTA 

SCH 

SSTC 

16.0 

40.8 

30.6 

TTA 

SCH 

SSTC 

59.9 

30.0 

100.0 

TTA 

SCH 

SSTC 

40.8 

16.0 

70.0 

TTA 

SCH 
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Table  34  Workload  Measurements  and  Estimates  (Continued) 


PRED 

PEN 

ACQ 

WPN 

CONFIG 

METHOD 

SSTC 

23.5 

39.2 

69.4 

TTA 

SCH 

SSTC 

59.9 

59.9 

83.6 

TTA 

SCH 

SSTC 

39.2 

52.2 

69.4 

TTA 

SCH 

SSTC 

59.0 

63.1 

27.6 

TTA 

SCH 

SSTC 

52.2 

52.2 

69.4 

TTA 

SCH 

SWAT 

7.99 

0.0 

87.57 

ABL 

PRO 

SWAT 

71.2 

87.57 

100.0 

ABL 

PRO 

SWAT 

21.23 

45.55 

61.92 

ABL 

PRO 

SWAT 

71.2 

83.63 

100.0 

ABL 

PRO 

SWAT 

61.92 

45.55 

61.92 

ABL 

PRO 

SWAT 

0.0 

7.99 

7.99 

TFTA 

PRO 

SWAT 

45.55 

37.56 

45.55 

TFTA 

PRO 

SWAT 

0.0 

45.55 

45.55 

TFTA 

PRO 

SWAT 

61.92 

45.55 

45.55 

TFTA 

PRO 

SWAT 

37.56 

57.98 

61.92 

TFTA 

PRO 

SWAT 

13.74 

45.55 

31.81 

TTA 

PRO 

SWAT 

13.74 

45.55 

74.35 

TTA 

PRO 

SWAT 

45.55 

13.74 

37.56 

TTA 

PRO 

SWAT 

37.56 

37.56 

83.63 

TTA 

PRO 

SWAT 

0.0 

21.73 

61.92 

TTA 

PRO 

SSI 

8.0 

8.0 

87.6 

ABL 

PRO 

SSI 

71.2 

71.2 

100.0 

ABL 

PRO 

SSI 

21.2 

45.6 

87.6 

ABL 

PRO 

SSI 

71.2 

84.9 

100.0 

ABL 

PRO 

SSI 

71.2 

45.6 

87.6 

ABL 

PRO 

SSI 

8.0 

8.0 

8.0 

TFTA 

PRO 

SSI 

45.6 

21.2 

45.6 

TFTA 

PRO 

SSI 

8.0 

45.6 

45.6 

TFTA 

PRO 

SSI 

71.2 

45.6 

45.6 

TFTA 

PRO 

SSI 

21.2 

58.0 

71.2 

TFTA 

P£0 

SSI 

21.2 

45.6 

31.8 

TTA 

PRO 

SSI 

21.2 

45.6 

84.9 

TTA 

PRO 

SSI 

45.6 

21.2 

21.2 

TTA 

PRO 

SSI 

21.2 

21.2 

84.9 

TTA 

PRO 

SSI 

8.0 

31.8 

71.2 

TTA 

PRO 

SSTA 

0.0 

0.0 

61.9 

ABL 

PRO 

SSTA 

61.9 

61.9 

74.4 

ABL 

PRO 

SSTA 

13.7 

35.6 

61.9 

ABL 

PRO 

SSTA 

61.9 

74.4 

74.4 

ABL 

PRO 

SSTA 

61.9 

61.9 

61.9 

ABL 

PRO 

SSTA 

0.0 

0.0 

0.0 

TFTA 

PRO 

SSTA 

35.6 

13.7 

35.6 

TFTA 

PRO 

SSTA 

0.0 

35.6 

35.6 

TFTA 

PRO 

SSTA 

61.9 

35.6 

35.6 

TFTA 

PRO 

SSTA 

13.7 

48.8 

61.9 

TFTA 

PRO 

SSTA 

13.7 

35.6 

21.7 

TTA 

PRO 
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Table  34  Workload  Measurements  and  Estimates  (Continued) 


PRED 

PEN 

acq 

WPN 

CONFIG 

METHOD 

SSTA 

13.7 

35.6 

74.4 

TTA 

PRO 

SSTA 

35.6 

13.7 

13.7 

TTA 

PRO 

SSTA 

13.7 

13.7 

74.4 

TTA 

PRO 

SSTA 

0.0 

21.7 

61.9 

TTA 

PRO 

MRMI 

0.0 

8.0 

87.6 

ABL 

PRO 

MRMI 

61.9 

71.2 

100.0 

ABL 

PRO 

MPMI 

13.7 

45.6 

87.6 

ABL 

PRO 

MRMI 

61.9 

84.9 

100.0 

ABL 

PRO 

MRMI 

61.9 

45.6 

87.6 

ABL 

PRO 

MRMI 

0.0 

8.0 

8.0 

TFTA 

PRO 

MRMI 

35.6 

21.2 

45.6 

TFTA 

PRO 

MRMI 

0.0 

45.6 

45.6 

TFTA 

PRO 

MRMI 

61.9 

45.6 

45.6 

TFTA 

PRO 

MRMI 

13.7 

58.0 

71.2 

TFTA 

PRO 

MRMI 

13.7 

45.6 

31.8 

TTA 

PRO 

MRMI 

13.7 

45.6 

84.9 

TTA 

PRO 

MRMI 

35.6 

21.2 

21.2 

TTA 

PRO 

MRMI 

13.7 

21.2 

84.9 

TTA 

PRO 

MRMI 

0.0 

31.8 

71.2 

TTA 

PRO 

MRMS 

8.0 

24.3 

87.6 

ABL 

PRO 

MENS 

71.2 

87.6 

100.0 

ABL 

PRO 

MRMS 

21.2 

59.9 

87.6 

ABL 

PRO 

MRMS 

71.2 

100.0 

100.0 

ABL 

PRO 

MRMS 

71.2 

59.9 

87.6 

ABL 

PRO 

MRMS 

8.0 

8.0 

24.3 

TFTA 

PRO 

MRMS 

45.6 

21.2 

60.0 

TFTA 

PRO 

MRMS 

8.0 

45.6 

60*0 

TFTA 

PRO 

MRMS 

71.2 

45.6 

60.0 

TFTA 

PRO 

MRMS 

21.2 

58.0 

87.6 

TFTA 

PRO 

MRMS 

21.2 

45.6 

46.8 

TTA 

PRO 

MRMS 

21.2 

45.6 

100.0 

TTA 

PRO 

MRMS 

45.6 

21.2 

37.6 

TTA 

PRO 

MRMS 

21.2 

21.2 

100.0 

TTA 

PRO 

MRMS 

8.0 

31.8 

87.6 

TTA 

PRO 

SWAT 

95.1 

40.8 

16.0 

ABL 

SCH 

SWAT 

59.9 

0.0 

100.0 

ABL 

SCH 

SWAT 

45.7 

70.0 

65.9 

ABL 

sea 

SWAT 

52.2 

41.8 

41.8 

ABL 

SCH 

SWAT 

59.9 

76.3 

59.9 

ABL 

SCH 

SWAT 

41.8 

72.0 

52.2 

ABL 

SCH 

SWAT 

59.0 

100.0 

52.2 

ABL 

SCH 

SWAT 

0.0 

53.2 

13.0 

ABL 

SCH 

SWAT 

26.2 

40.8 

16.0 

TFTA 

SCH 

SWAT 

70.4 

0.0 

45.7 

TFTA 

SCH 

SWAT 

65.9 

45.2 

65.9 

TFTA 

SCH 

SWAT 

41.8 

28.8 

41.8 

TFTA 

SCH 
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Table  34  Workload  Measurements  and  Estimates  (Continued) 


PRED 

PEN 

ACQ 

WPN 

CONFIG 

METHOD 

SWAT 

76.3 

76.3 

59.9 

TFTA 

SCH 

SWAT 

28.8 

52.2 

52.2 

TFTA 

SCH 

SWAT 

72.0 

63.1 

0.0 

TFTA 

SCH 

SWAT 

52.2 

0.0 

0.0 

TFTA 

SCH 

SWAT 

16.0 

40.8 

1.4 

TTA 

SCH 

SWAT 

59.9 

0.0 

76.3 

TTA 

SCH 

SWAT 

40.8 

45.2 

40.8 

TTA 

SCH 

SWAT 

13.0 

28.8 

41.8 

TTA 

SCH 

SWAT 

59.9 

59.9 

59.9 

TTA 

SCH 

SWAT 

28.8 

52.2 

52.2 

TTA 

SCH 

SWAT 

48.5 

63.1 

0.0 

TTA 

SCH 

SWAT 

52.2 

52.2 

41.8 

TTA 

SCH 

SSTA 

64.5 

39.3 

14.6 

ABL 

SCH 

SSTA 

29.9 

0.0 

46.3 

ABL 

SCH 

SSTA 

44.2 

39.3 

64.5 

ABL 

SCH 

SSTA 

41.8 

41.8 

41.8 

ABL 

SCH 

SSTA 

29.9 

46.3 

29.9 

ABL 

SCH 

SSTA 

41.8 

61.5 

41.8 

ABL 

SCH 

SSTA 

48.5 

72.4 

41.8 

ABL 

SCH 

SSTA 

0.0 

41.8 

13.0 

ABL 

SCH 

SSTA 

24.7 

39.3 

14.6 

TFTA 

SCH 

SSTA 

40.4 

0.0 

15.7 

TFTA 

SCH 

SSTA 

64.5 

14.6 

64.5 

TFTA 

SCH 

SSTA 

41.8 

28.8 

41.8 

TFTA 

SCH 

SSTA 

46.3 

46.3 

29.9 

TFTA 

SCH 

SSTA 

28.8 

41.3 

41.8 

TFTA 

SCH 

SSTA 

61.5 

52.6 

0.0 

TFTA 

SCH 

SSTA 

41.8 

0.0 

0.0 

TFTA 

SCH 

SSTA 

14.6 

39.3 

0.0 

TTA 

SCH 

SSTA 

29.9 

0.0 

46.3 

TTA 

SCH 

SSTA 

39.3 

14.6 

39.3 

TTA 

SCH 

SSTA 

13.0 

28.8 

41.8 

TTA 

SCH 

SSTA 

29.9 

29.9 

29.9 

TTA 

SCH 

SSTA 

28.8 

41.8 

41.8 

TTA 

SCH 

SSTA 

48.5 

52.6 

0.0 

TTA 

SCR 

SSTA 

41.8 

52.2 

41.8 

TTA 

SCH 

SWAT 

0*0 

7.99 

61.92 

ABL 

SWAT 

SWAT 

45.55 

100.0 

87.57 

ABL 

SWAT 

SWAT 

0.0 

74.35 

100.0 

ABL 

SWAT 

SWAT 

37.56 

87.57 

45.55 

ABL 

SWAT 

SWAT 

74.35 

45.55 

45.55 

ABL 

SWAT 

SWAT 

7.99 

45.55 

87.57 

TiTA 

SWAT 

SWAT 

0.0 

45.55 

45. 55 

TFTA 

SWAT 

SWAT 

0.0 

13.74 

61.92 

TFTA 

SWAT 

ftfAT 

57.98 

37.56 

31.81 

TFTA 

SWAT 

SWAT 

75.64 

57.98 

100.0 

TFTA 

SWAT 

Table  34  Workload  Measurements  and  Estimates  (Continued) 


PRED 

PEN 

ACQ 

WPN 

CONI xG 

METHOD 

SWAT 

37.56 

45.55 

23.82 

TTA 

SWAT 

SWAT 

0.0 

45.55 

61.92 

TTA 

SWAT 

SWAT 

31.81 

13.74 

37.56 

TTA 

SWAT 

SWAT 

37.56 

37.56 

63.21 

TTA 

SWAT 

SWAT 

0.0 

38.1 

100.0 

TTA 

SWAT 
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